Using Tests

This chapter provides guidelines for using locally and externally developed tests for assessment. Tips for planning and developing a test as well as for analyzing its quality are included.

Definitions (Test and Standardized Test)

Tests are commonly used in association with cognitive goals, to review student achievement with respect to a common body of knowledge associated with a discipline or practice.

A standardized test is one in which the initial construction, as well as conditions for administration and scoring, have a uniform procedure. This ensures that scores can be interpreted in a consistent manner from one administration to the next. Standardized tests are usually designed by test development specialists, either internally or externally.

Appropriate Use of Tests

Tests should be used when:

  • a valid and reliable test is available.
  • student acquisition of knowledge or ability to process and use knowledge is to be demonstrated (i.e., the outcome of interest is cognitive in nature).
  • student knowledge about a wide range of content is to be evaluated (in survey and capstone courses).
  • multiple observations of content-related knowledge are needed (math and foreign languages).
  • more resources are available for constructing the assessment instrument than for scoring and reporting results.
  • a large group is being assessed.

Tests should not be used when:

  • there is disagreement about the choice, design, or content of the test to be used.
  • the scoring of the test is not reliable or valid.
  • the content of the test does not match the goals of the department.
  • the number of participants is small.

Advantages and Disadvantages of Tests

Advantages

  • Well-constructed tests sample student knowledge with efficiency and reliability. The test given can determine what many students know in a brief period of time.
  • The repeated use of a test will provide a means of comparison between different student groups or the same group over time. This type of testing practice provides reviewers with a rich context for evaluation, decision making, and making recommendations.

Disadvantages

  • Tests lack flexibility. Because tests are usually designed by organizations and companies outside the department, the content of the test is predetermined and cannot be modified to match department goals or curriculum.
  • Tests can be expensive. Costs associated with purchasing tests and processing results must be considered during assessment planning.

Planning Tests

One of the most important planning decisions is the choice of test. Tests can be standardized (purchased tests that are developed by a testing company) or created locally by a department or committee. The match between test and assessment purpose is critical.

Features of a Good Test

A good test:

  • has a well-defined purpose or intent.
  • has a foundation based on a set of written goals and objectives.
  • shows evidence that the test’s purpose was achieved (reliability and validity information about test items, the test as a whole, and the relationship between test scores and other indices of academic performance).
  • contains detailed scoring procedures that allow for specific interpretations and feedback to those tested and to those making decisions.

Analyzing Test Data

Test data are scaled and then analyzed using a number of techniques including descriptive statistics and/or multivariate analysis (Palomba & Banta, 1999). Descriptive statistics techniques (maximum, mean, median, minimum, and mode) describe the characteristics of the data. Multivariate analysis requires using regression methods and analyzing variances within the data. Multivariate analysis typically uses preliminary testing and background characteristics of participants to attempt to predict test scores.

Frequently-Asked Questions

What are some strategies for creating a department test for assessment purposes?

One common practice is to develop and adopt common pre-tests and post-tests in courses with multiple sections. Items for common tests can be culled from existing exams. Another practice is to determine a portion of each unit exam, a specific set of items, which will be scored for program assessment and for individual evaluation.

This practice is sometimes referred to as course-embedded testing (Palomba & Banta, 1999, p. 13). It is important to notify students how this testing will affect assignment of their grades.

What are the basic steps in developing a test?

Seven sequential steps are recommended:

  1. Determine outcomes to be measured.
  2. Develop test blueprint.
  3. Write test items.
  4. Review, critique, and edit items.
  5. Pilot test items.
  6. Obtain reliability and validity data.
  7. Revise, reuse, and report.

How is using a test for assessment different from using a test in the classroom?

Generally, instructors develop their own classroom tests, making all decisions about when and how to construct, administer, and score the test and report results. The tests are constructed without formality or documentation for the purpose of assigning grades related to individual student learning. When tests are used for assessment, planning, implementing, and using results becomes a group effort, a shared set of decisions and responsibilities. Consensus is emphasized. Some additional planning time, communication, and record keeping will be needed. Test performance is generally used along with other information to describe group achievement and is independent of grading.

References

Palomba, C. A., & Banta, T. W. (1999). Assessment essentials: Planning, implementing, and improving assessment in higher education. San Francisco: Jossey-Bass.