Overview of Student Learning Outcomes Assessment

By taking ownership of assessment and developing an internally driven core process, colleges and universities can profile their students' learning within institutional educational practices and intentions. Moreover, within this context, assessment becomes a means to examine its educational intentions on its own terms within the complex ways that humans learn and within the populations an institution serves. (Maki, 2004, p.15)

Miami University's 2019 strategic plan documents notes in its statement of purpose:

Now is the time for Miami to transform for a new era, building an adaptive, responsive and financially sustainable foundation, with students immersed in academic and co-curricular experiences that prepare them to succeed in today's interconnected world. We will welcome students, faculty and staff of all backgrounds to a united Miami community, always learning and committed to a culture of investment and continuous improvement. (p. 4)

The Higher Learning Commission, Miami's regional accreditation, association, requires universities to assess student learning outcomes within the majors or degree programs as well as general education. The relevant accreditation criteria are listed below:

3.A. The institution's degree programs are appropriate to higher education.
  1. Courses and programs are current and require levels of performance by students appropriate to the degree or certificate awarded.
  2. The institution articulates and differentiates learning goals for its undergraduate, graduate, post-baccalaureate, post-graduate, and certificate programs.
  3. The institution's program quality and learning goals are consistent across all modes of delivery and all locations (on the main campus, at additional locations, by distance delivery, as dual credit, through contractual or consortia arrangements, or any other modality).
4.B. The institution demonstrates a commitment to educational achievement and improvement through ongoing assessment of student learning.
  1. The institution has clearly stated goals for student learning and effective processes for assessment of student learning and achievement of learning goals.
  2. The institution assesses achievement of the learning outcomes that it claims for its curricular and co-curricular programs.
  3. The institution uses the information gained from assessment to improve student learning.
  4. The institution's processes and methodologies to assess student learning reflect good practice, including the substantial participation of faculty and other instructional staff members.

Thus, due to both internal commitments and external requirements, the University must have assessment of student learning conducted at the departmental, divisional and institutional levels. Assessment is important not only because we are required to do it, but also because it is the right thing to do.

The University of Delaware (n.d.) has articulated the following benefits of assessment:

  • increasing our confidence that we are putting our time and resources into activities that we value as an institution
  • increasing our confidence that we are allocating resources to areas that are producing the outcomes we value
  • gathering and using data that will enable us to make decisions that lead to improved instruction, stronger curricula, and effective and efficient policies
  • strengthening our ability to say that our graduates are well-prepared to succeed in their future endeavors
  • having ready access to data that will satisfy the requirements of accrediting agencies and funding agencies, and will inform various accountability driven conversations
  • gathering and using data that will strengthen arguments for increased funding and/or resource allocations to areas that are producing valued outcomes
  • increasing the effectiveness of our communications about the value of a Miami University education

The following principles of good practice, originally released at the American Association for Higher Education's Assessment Forum, provide important guidance for Miami's assessment efforts:

  • The assessment of student learning begins with educational values.
  • Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time.
  • Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes.
  • Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes.
  • Assessment works best when it is ongoing, not episodic.
  • Assessment fosters wider improvement when representatives from across the educational community are involved.
  • Assessment makes a difference when it begins with issues of use and illuminates questions that people really care about.
  • Assessment is more likely to lead to improvement when it is part of a larger set of conditions that promote change.
  • Through assessment, educators meet responsibilities to students and to the public.
  • Educational programs, in order to be successful, require full engagement of faculty and staff members in the conversations about, and the design and practice of, student learning outcomes assessment.

(Hutchings, Ewell, & Banta, 2012)

Responsibility for maintaining and improving an effective assessment program rests with several sources:

University Responsibilities

The University Assessment Council is responsible for overseeing assessment of student learning at Miami University. Its mission is “to promote a culture of assessment of student learning outcomes at Miami University in which data are used to improve the quality of the educational experience." Toward that end, the committee:

  • advises on policies, procedures, faculty development efforts, and best practices relating to assessment of student learning outcomes.
  • annually collects assessment reports from departments and programs, including academic support units.
  • annually reviews the effectiveness of plans, reports, and practices for assessing student learning at the University, department and program levels.
  • communicates to Miami faculty and staff about assessment of student learning (e.g., Assessment Briefs).
  • assists divisional committees in evaluating assessment plans and reports effectively prior to a department's academic program review.
  • provides guidance to Liberal Education Council on the assessment of the Global Miami Plan.
  • assists with other initiatives involved in the assessment of student learning as requested by the Provost.

The committee consists of the following members: Assistant Provost for Institutional Research and Effectiveness (chair); Director of Center for Teaching Excellence; Director of Liberal Education; Associate Dean of Students; Associate Dean of University Libraries; and one member (typically an associate dean) from each of the academic divisions who oversees assessment and/or curricular initiatives in the division.

Departmental Responsibilities (Academic, Academic Support, and Co-Curricular)

The departmental procedure for assessing student learning in degree programs is as follows:

Appoint departmental assessment contact(s) who is/are responsible for coordinating assessment of all majors, degree programs, and “free-standing” certificates. Submit name to William Knight, Assistant Provost, for Institutional Research and Effectiveness, who will provide assistance and support.

Determine an annual submission date. Each department will be expected to submit its new plan (when developing or revising) and its annual assessment report(s) once each year. Departments may select either end of June or end of December for the annual submission date. Materials should be submitted annually to Dr. Knight by the selected date.

Develop or revise plan for assessing at least three student learning outcomes for each degree program, major and free-standing certificate in department. Try to select at least one introductory and one capstone/culminating course to collect and assessment data. If the program is offered in in-person and online modes, be sure to compare findings. Submit to Dr. Knight by the annual submission date (when developing or revising).

Collect data (student work) from the appropriate courses and other learning opportunities annually (or as often as possible) during appropriate semesters or times. Assess using the rubric(s), scoring guides or other instruments identified in your assessment plan.

Share assessment progress and findings annually with faculty or staff colleagues in the department. Once sufficient data are available to develop significant findings, identify steps for improvement that are aligned with findings.

Create an annual report that summarizes the plan for assessment (outcomes, method of assessment), significant findings from the assessment, and plans for improvement based upon assessment findings. If there is insufficient evidence to lead to substantive conclusions, submit an annual report summarizing where you are in the data-gathering process. Submit report to Dr. Knight by your selected submission date (end of June or December) each year.

Each year, review the brief feedback report offered by Dr. Knight and be sure to address recommendations during the next reporting cycle.

Prior to your program review, carefully review the evaluation report of the divisional committee as well as report of prior program review teams. Meet with your representative on the University Assessment Council to revise the assessment plan to address feedback offered in reports.

Divisional Responsibilities

Each division's curriculum and/or assessment committee is responsible for reviewing each department's assessment activity by the end of April the year prior to when the department undergoes program review. The procedure is summarized below.

  1. University Assessment Council members train all appropriate divisional committees on how to evaluate departmental assessment activity each year. The Assistant Provost provides committees with the annual assessment reports of the department that have been created since the last program review.
  2. Committees review and evaluate departments' assessment activity and creates summary reports. Committees submit reports to William Knight, Assistant Provost for Institutional Research and Effectiveness.
  3. The report is included in all program review materials and evaluated by the external and internal review teams as part of the normal program review process.

Liberal Education Office Responsibilities

The Revision of the Global Miami Plan approved by University Senate in 2014 included a brief description of a plan for assessing student learning. The assessment plan noted that the “two signature competencies of written communication and critical thinking will be formally assessed."

The Office of Liberal Education in collaboration with Liberal Education Council and the University Assessment Council is responsible for overseeing assessment of student learning in the Global Miami Plan.

Office of Institutional Research and Effectiveness Responsibilities

The Assistant Provost for Institutional Research and Effectiveness provides leadership and support for all aspects of student learning assessment. Upon request the Office of Institutional Research and Effectiveness provides information (e.g., courses taken, entrance examination scores, institutional survey results) to contextualize and enrich assessment results as well as assistance with surveys and data analysis.

Support for Departments and Divisions

  • Assessment Resources Website, which includes an overview of assessment of student learning, templates for and samples of departmental assessment plans and reports, sample rubrics and curriculum maps, templates and samples of evaluation reports of departmental assessment activity, etc.
  • Annual Feedback on Assessment Reports to offer departments further ideas and support for continuous improvement.
  • Training Workshops for Divisional Committees, which offers guidelines for evaluating departmental assessment reports as well as an interactive portion in which participants practice evaluating sample documents from an imaginary department.
  • Bi-Monthly Assessment Briefs, which provide success stories, tips and assessment findings from departments and other units across the University.
  • Assessment Projects, which focus on special topics of interest, such as comparing the quality of learning in a full-term versus a compressed delivery course.

Glossary of Assessment Terms

Assessment

is the systematic collection, review, and use of information about educational programs undertaken for the purposes of improving student learning and development (Palomba & Banta, 1999). The purpose of assessment is to provide information about the student learning and development that occurs as a result of a program. A “program” may be any activity, project, function, or policy that has an identifiable purpose or set of objectives.

Curriculum Mapping

The process of aligning courses/activities with program/major level goals and objectives, often done systematically with faculty/staff involvement. Curriculum mapping is a process for recording what content and skills are actually taught in a class, activity, or program.

Measure

Measurement refers to the process by which the attributes or dimensions of some physical object (e.g., student) are determined. In the context of assessment of student learning or development, measurement can involve a combination of qualitative and quantitative information to determine levels or qualities of student learning and development. The word measure is also intended may address the type or level of program activities conducted (process), the direct products and services delivered by a program (outputs), and/or the results of those products and services (outcomes).

Direct Measures

Direct measures require students to demonstrate their knowledge and skills. They provide tangible, visible and self‐ explanatory evidence of what students have and have not learned as a result of a course, program, or activity (Suskie, 2009; Palomba and Banta, 1999).

Authentic

Based on examining genuine or real examples of students' work. Work that closely reflects goals and objectives for learning. Authentic assessment reveals something about the standards that are at the heart of a subject; asking students to use judgment and innovation as they “do” and explore the subject. (Wiggins, 1989 as in Palomba & Banta, 1999).

Embedded

Program, general education, or institutional assessments that are embedded into course work. In other words, they are course assessments that do double duty, providing information not only on what students have learned in the course but also on their progress in achieving program or organizational goals. Because embedded assessment instruments are typically designed by faculty and staff, they match up well with local learning goals. They therefore yield information that faculty and staff value and are likely used to improve teaching and learning (Suskie, 2009).

Portfolio Assessment

A type of performance assessment in which students' work is systematically collected and reviewed for evidence of student learning. In addition to examples of their work, most portfolios include reflective statements prepared by students. Portfolios are assessed for evidence of student achievement with respect to established student learning outcomes and standards (Palomba & Banta, 1999).

Full-Cycle Assessment

The complete process of developing learning goals (What do we want students to be able to do when they complete our program?), collecting assessment evidence (How well are students achieving these goals, and what factors influence their learning?), and reflecting upon implications and taking-action based on assessment results (How can we use the information to improve student learning?). Two additional steps in the assessment process that may be useful include mapping learning goals to the curriculum (Where do students have the opportunity for learning?) and stating expectations (What is the expected level of performance?)

Indirect Measures

Assessments that measure opinions or thoughts about students' or alumni's own knowledge, skills, attitudes, learning experiences, perception of services received or employers' opinions. While these types of measures are important and necessary they do not measure students' performance directly. They supplement direct measures of learning by providing information about how and why learning is occurring (Hansen, 2011).

Focus groups

A group selected for its relevance to an evaluation that is engaged by a trained facilitator in a series of discussions designed for sharing insights, ideas, and observations on a topic of concern to the evaluation (National Science Foundation, 2010).

Interviews

Occur when researchers ask one or more participants general, open-ended questions and records their answers (Creswell, 2008).

Surveys

A survey is a method of collecting information from people about their characteristics, behaviors, attitudes, or perceptions. Surveys most often take the form of questionnaires or structured interviews (Palomba &Banta, 1999). General definition: an attempt to estimate the opinions, characteristics, or behaviors of a particular population by investigation of a representative sample.

The Higher Learning Commission (HLC)

One of six regional institutional accreditors in the United States. The Higher Learning Commission accredits degree-granting post-secondary educational institutions in more than 1,000 colleges and universities in nineteen states. The states are Arkansas, Arizona, Colorado, Iowa, Illinois, Indiana, Kansas, Michigan, Minnesota, Missouri, North Dakota, Nebraska, Ohio, Oklahoma, New Mexico, South Dakota, Wisconsin, West Virginia, and Wyoming.

Rubric

A set of categories that define and describe the important components of the work being completed, critiqued, and assessed. Each category contains a gradation of levels of completion or competence with a score assigned to each level and a clear description of what criteria need to be met to attain the score at each level.

Student Learning Outcomes

Specify what students will know, be able to do, or be able to demonstrate when they have completed or participated in academic program(s) leading to certification or a degree. Outcomes are often expressed as knowledge, skills, attitudes, behaviors, or values. A multiple methods approach is recommended to assess student learning outcomes indirectly and directly. Direct measures of student learning require students to demonstrate their knowledge and skills. They provide tangible, visible and self-explanatory evidence of what students have and have not learned as a result of a course, program, or activity (Suskie, 2009; Palomba & Banta, 1999).

Value Added

The increase in learning that occurs during a course, program, or undergraduate education. Can either focus on the individual student (how much better a student can write, for example, at the end than at the beginning) or on a cohort of students (whether senior papers demonstrate more sophisticated writing skills in the aggregate than freshmen papers). Requires a baseline measurement for comparison (Leskes, 2002).

(adapted from the Glossary of Assessment Terms compiled by the IUPUI Advanced Practices Committee)

References

Creswell, J. W. (Ed.). (2008). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (3rd Ed.). Upper Saddle River, New Jersey: Pearson Prentice Hall.

Hansen, M. (2011). Direct and Indirect Measures of Student Learning. Minutes: Program Review and Assessment Committee (March 11, 2001). IUPUI.

Leskes, A. (2002). Beyond confusion: An assessment glossary [Electronic version]. Association of American Colleges and Universities (AACU): Peer Review, Winter / Spring, 2002. Retrieved May 6, 2011from: http://assessment.uconn.edu/docs/resources/Andrea_Leskes_Assessment_Glossary.pdf.

Hutchings, P., Ewell, P, & Banta, T. (2012). AAHE Principles of Good Practice: Aging Nicely. Retrieved from http://www.learningoutcomesassessment.org/PrinciplesofAssessment.html.

IUPUI Advanced Practices Committee. (n.d.). Glossary of assessment terms. Retrieved from https://planning.iupui.edu/assessment/prac-files/glossary/glossary-terms.pdf.

Maki, P. L. (2004). Assessment for learning: Building a sustainable commitment across the institution. Sterling, VA: American Association of Higher Education and Stylus Publishing.

Miami University. (2019). Miami University's strategic plan. Retrieved from https://miamioh.edu/_files/documents/about-miami/president/strategic-plan/strategic-plan-06-28-19_508.pdf

National Science Foundation. (2010). The 2010 user-friendly handbook for project evaluation. United States. Retrieved April 22, 2011 from: http://caise.insci.org/uploads/docs/TheUserFriendlyGuide.pdf

Palomba, C. A. & Banta, T. W. (1999). Assessment essentials: Planning, implementing, and improving assessment in higher education. San Francisco, CA: Jossey-Bass.

Suskie, L. (2009). Assessing student learning: A common sense guide (2nd ed.). San Francisco, CA: Jossey-Bass.

University of Delaware, Office of Educational Assessment. (n.d.). Student learning outcomes assessment manual.

Wiggins, G. (1989). A true test: Toward a more authentic and equitable assessment. Phi Delta Kappan, 70, pp. 703-713.