Assessment Brief #83 - Miami Plan Assessment via Program Level Assessment Part 2

Assessment Logo: assessment-revision-outcomes

May 2014

Introduction and Purpose

In the fall of 2011, University Senate approved a new plan that would embed Global Miami Plan (GMP) assessment into program-level assessment. This decision recognized that GMP goals are promoted not only by GMP courses, but also in courses within majors. In the current study, we examine the extent to which programs are assessing GMP goals by assessing program level student learning outcomes (SLOs) and compare that to the earlier method of tracking assessment of GMP goals via reporting individual GMP courses at program review (as reported in Assessment Brief #82).

Procedure

We examined program assessment reports (between 2011 and 2013) to find out which program-level SLOs have been assessed, and thus, which GMP goals have been assessed. We asked program representatives to tell us the linkages between each of their SLOs and the four (critical thinking, understanding contexts, engaging other learners, reflecting & acting ) GMP goal(s), noting that each SLO might be related to one or more GMP goals, or none.

Sample and Analysis

Forty-six (out of 113, or 40.7%) undergraduate degree programs participated: College of Arts and Science 22 reports; Farmer School of Business 3; College of Engineering and Computing 7; College of Education, Health and Society 4; College of Creative Arts 7; and College of Professional Studies and Applied Sciences 3. We examined their assessment reports to find out which GMP goals were assessed, in which courses (100-level, 200-300, 400-level), and the type of assessment conducted (i.e., faculty reflection-Tier I, student perception -Tier II, or assessment of student work-Tier III). Each program had 36 total opportunities to assess GMP goals (i.e., 3 tiers x 3 course levels, by 4 GMP). If a 400-level course assessed critical thinking by means of student work, a tally was entered into the appropriate cell of Table 1 below. The average number of tally marks (assessments) per department was about 15 (14.7), with a range of 2-36.

GMP Goals Assessment by Tier and Goal grouped by Course Level

  • 100 Level
    • Critical Thinking
      • Tier I - 11
      • Tier II - 6
      • Tier III - 12
      • Total - 29
    • Understanding Contexts
      • Tier I - 11
      • Tier II - 6
      • Tier III - 12
      • Total - 29
    • Engaging With Other Learners
      • Tier I - 9
      • Tier II - 7
      • Tier III - 11
      • Total - 27
    • Reflecting & Acting
      • Tier I - 11
      • Tier II - 7
      • Tier III - 13
      • Total - 31
    • Total
      • Tier I - 42
      • Tier II - 26
      • Tier III - 48
      • Total - 116
  • 200-300 Level
    • Critical Thinking
      • Tier I - 24
      • Tier II - 9
      • Tier III - 26
      • Total - 59
    • Understanding Contexts
      • Tier I - 24
      • Tier II - 9
      • Tier III - 26
      • Total - 59
    • Engaging With Other Learners
      • Tier I - 19
      • Tier II - 10
      • Tier III - 21
      • Total - 50
    • Reflecting & Acting
      • Tier I - 23
      • Tier II - 10
      • Tier III - 24
      • Total - 57
    • Total
      • Tier I - 90
      • Tier II - 38
      • Tier III - 97
      • Total - 225
  • 400 Level
    • Critical Thinking
      • Tier I - 35
      • Tier II - 15
      • Tier III - 39
      • Total - 89
    • Understanding Contexts
      • Tier I - 34
      • Tier II - 15
      • Tier III - 38
      • Total - 87
    • Engaging With Other Learners
      • Tier I - 31
      • Tier II - 14
      • Tier III - 33
      • Total - 78
    • Reflecting & Acting
      • Tier I - 33
      • Tier II - 15
      • Tier III - 35
      • Total - 83
    • Total
      • Tier I - 133
      • Tier II - 59
      • Tier III - 145
      • Total - 337

Summary and Comparison to the Earlier Method

In the previous study on how GMP goals were assessed at Miami via program review of GMP courses, assessment of student work was the least common method, whereas in this study it was most common. There were 290 assessments of student work, vs. 265 faculty reflections and 123 student perceptions. When GMP goals were assessed in GMP courses, all three course levels were comparable, whereas when GMP goals were assessed via program-level SLOs, assessment increased with course level. When GMP goals were assessed in GMP courses, the goal of critical thinking was by far the most frequently assessed, whereas when GMP goals were assessed via program-level SLOs, all GMP goals were comparable, although critical thinking (177) and understanding context had a slight edge (175), followed by reflecting and acting ( 171), and the goal of engaging other learners (155) slightly lagged behind. Thus the two methods of tracking GMP assessment give complementary pictures of the ways that Miami faculty promote and assess GMP goals.