2015 Program Grant Winner

Tony Cimasko

English (ESL Composition)


Programmatic Writing Assessment of English 108


The College Composition program in the English department is seeking funding to conduct an outcomes-based, programmatic assessment of student writing in English 108, US Cultures and Composition for Second Language Writers. English 108 is a 4-credit hour first-year writing course designed for international students who speak and write English as a second language at a minimal level for full university admission to develop basic writing and rhetorical comprehension. The course also provides foundational instruction on US academic norms and aspects of US cultures; students who pass ENG 108 earn Miami Plan Global credits. In the 2014–2015 school year, English 108 enrolled 316 students on the Oxford campus. Enrollment of 108 has been increasing substantially over the past several years and is expected to expand even further as the number of international students attending Miami continues to grow.

Our programmatic writing assessment aims:

  • To ensure curricular consistency in meeting the common course writing outcomes of English 108.
  • To provide recommendations and guidance for ways to revise our curricular materials and professional development programs to help instructors and students better achieve our course outcomes.
  • To offer a substantial professional development opportunity in rubric-based writing assessment for our continuing ENG 108 instructors.
  • To assess the effectiveness of the writing and cultural components of the curriculum.
  • To assess transfer of knowledge from ENG 108 to ENG 109 and other academic contexts.
  • To further refine our course outcomes to ensure that they are aligned with university-wide expectations.

Building on and Extending Past Programmatic Writing Assessments

In 2013, we conducted our first programmatic assessment of ENG 109, Composition and Rhetoric for Second Language Writers. This assessment was invaluable in helping us to see the successes and limitations of ENG 109 on its own terms, to understand how well ENG 109 was “translating” the experience of the first language-oriented ENG 111 to second language users, and how our curriculum and materials can be most productively revised to better prepare students in what is their last ESL-only classroom context at this university.

Although ENG 108 has existed for nearly as long as ENG 109 and serves a student population that is nearly as large (more than 90 percent of all international undergraduates enroll in ENG 108), we have not yet found the time or funds to properly conduct a program-wide, direct assessment of the class. The ENG 108 curriculum was revised several years ago in order to have the course count for Miami Global credit. Furthermore, we have been refining the 108 curriculum over the past two years to make it more challenging and to more clearly distinguish it from courses in the American Culture and English program, which now serves students who were formerly at the lower level of ENG 108. In order to ensure that the curriculum of English 108 is meeting the writing and cultural outcomes of the course, the proposed programmatic assessment of student writing in English 108 is necessary.

Assessment Process

Throughout this semester, we have been collecting randomly selected student writing portfolios) from all sections of English 108. These digital portfolios contain all the formal writing assignments students completed in the course; we will remove instructor and student names from these portfolios before conducting the assessment. In late April 2015, we intend to recruit a team of six experienced ESL composition instructors to conduct a programmatic assessment of 45 to 60 student portfolios from at least 8 different sections of English 108. Two readers, who will have been trained and aligned in employing our outcomes-based rubric for assessment, will read each portfolio.

In addition to quantitatively scoring the portfolios using an outcomes-based rubric, the assessment team members will also record qualitative observations about how well each assignment in our curriculum appears to be meeting course outcomes as well as about any common problems or issues they notice in student writing. After the qualitative and quantitative assessment reading is complete, the whole assessment team will meet to engage in dialogue about recommendations for curricular change and instructor development.

Using Programmatic Assessment to Transform Curriculum and Instructor Training

The assessment coordinators will prepare a detailed assessment report and distribute it among program faculty and department administration. As with our prior assessment of ENG 109, we will publish key assessment findings in both our print and online Teacher’s Guide, which is given to all composition faculty members each year. We will coordinate efforts with Teacher’s Guide editors to ensure that guide contains redesigned curricular materials (writing prompts, rubrics, lesson plans) designed to address areas of concern identified by the assessment team. We also will use the assessment findings to guide our ongoing professional development, through regular shared observations among members of the ESL Composition faculty. In particular, we plan to ask members of assessment team to lead a workshop for all members of the ESL Composition committee about how they have used the assessment findings to refine their own teaching of ENG 108. Furthermore, the committee will use the assessment results to collaboratively develop new curricular materials throughout the year to improve student writing in the course.

Collaboration with the Howe Center

We will also share our assessment findings with Howe Center staff, as many English 108 students visit the center both for consultations and workshops. In addition to sharing our report with the Howe Center, we also hope to create workshops for Howe center staff that are oriented around key assessment findings to foster discussion about how individual consultations and workshops can help students better meet our course outcomes; furthermore, as we work to design new curricular materials in light of our assessment findings, we plan to collaborate with Howe Writing consultants to revise our writing assignment prompt templates in ways that better make our expectations clear for students.

In the 2015-16 school year, Tony Cimasko (the ESL Composition coordinator) also hopes to collaborate with the Howe Center in conducting research about what aspects of US cultures are most common in writing assignments across the disciplines at Miami, gathering data that could help both the Composition Program and the Howe Center ensure that we are teaching what cultural knowledge is most needed for student academic success. This research would likely involve a survey of students and their instructors.

Schedule

  • LATE APRIL 2015: An assessment team of experienced composition instructors is selected through a competitive process. The ESL composition committee revises the ENG 109 assessment outcomes rubric to reflect the particular language and culture learning objectives of ENG 108.
  • EARLY MAY 2015: Assessment coordinators anonymous and randomly selected portfolios to prepare them for reading.
  • MID MAY 2015: Assessment team meets for 10 hours over two days to train and align readers, qualitatively and quantitatively assess portfolios, and to discuss findings and make recommendations.
  • LATE MAY AND EARLY JUNE 2015: Assessment team members draft final report and coordinate with the composition publications committee to integrate their findings and a guide to making recommended changes into the upcoming edition of Teacher’s Guide.
  • SUMMER 2015 – SUMMER 2016: Program administrators design and implement professional development programs to disseminate curricular modifications recommended by the assessment team. The Summer 2016 Teacher’s Guide committee continues to design additional curricular materials to implement the recommendations of the assessment team.

Team Members

  • Dr. Tony Cimasko (assessment team leader), Lecturer in English and Coordinator of ESL Composition
  • 6 assessment readers (all experienced composition instructors to be selected in late April)

Proposed Budget

  • Assessment team leader (Tony Cimasko): $1000
  • Portfolio assessment readers: $1,800 (6 @ $300)
  • Miscellaneous costs: $150
  • Total: $2,950 (from Howe Center Grant)