Standard Test Scoring

The standard output you will receive from a test scoring request will include the following:

• Test Messages
• Overall Test Statistics
• Distribution Table
• Plot of Test Distribution
• Choice Analysis (if appropriate)
• 10-Choice Analysis (if appropriate)
• High-Low Item Analysis (Type 2)
• Student Listing
• List of Students Who Left Questions Blank (if appropriate)

Additional Output Available

• Separate Section Listing
• Individual Student Output
• Test Scramble (if appropriate)

Computerized Grade Books (accessed from https://www.admin.miamioh.edu/cfapps/test_score)

• Data Set Name for the SCO Data
• Data Set Name for the RAW Data
• Data Set Name for the CSV Data, which can be uploaded to a Learning Management System (LMS)

Test Messages

Any special situations that were encountered will be displayed here. For example, the message STUDENT NOT CURRENTLY ENROLLED IN COURSE indicates a problem with either the test form or the student’s enrollment status. Possible reasons include that the course was indicated incorrectly, course was entered into the program incorrectly, student ID was entered incorrectly, or students may be auditing and not officially enrolled in the course. Tests with invalid IDs are scored and the invalid name appears on the listing.

Overall Test Statistics

Test Mean: The test mean is the average score that the class obtained. If applicable, the mean will reflect the penalty fractions.

Test Standard Deviation: This statistic is a measure of the variability of scores from the mean.

Test Mode: The test mode is the most frequently obtained score. In the case where multiple modes exist, the lowest score is taken as the mode. Other modes can be found by looking at the distribution table.

High Score: This is the highest score on the test.

Low Score: This is the lowest score on the test.

Range: The range is the numerical difference between the highest and lowest scores on the test.

Possible Points: This statistic is the score that a perfect test would receive. Normally, possible points will equal the number of questions on the test; however, when an instructor decides to omit questions, the possible points statistic is not equal to the number of questions, but is equal to the number of questions minus the number of questions deleted. If the number of questions and the total number of possible points do not match, then answer(s) have been left blank on the test key. If an answer is left blank on the key, the test scoring program assumes that the question is to be deleted.

Total Tests Graded: This is the number of answers sheets successfully graded.

Distribution Table: The distribution table gives an overall view of class performance on a test. By looking at the distribution table, one can see how many students obtained the scores between the lowest and highest as well as the percent of the class that received an equal, lower, and higher score.

Plot of Test Distribution: A histogram of the distribution table provides a visual indication of class performance. The numbers at the extreme left of the graph correspond to scores; the numbers immediately preceding the rows of asterisks correspond to the number of students receiving that score.

5-Choice Analysis and 10-Choice Analysis

The purpose of an item analysis is:

1. to determine how efficient the individual test questions function, and
2. to provide information about the areas that are particular sources of difficulty for students.

Type 1 Item Analysis

A Type 1 item analysis reveals how the class responded to each question on the test. The column under the heading, “No.” indicates the question number. The columns under the headings, (1), (2), (3), (4), etc., indicate how many students marked this response as the answer to the question number at the extreme left. The item analysis also indicates how many students left the question blank. It should be noted that a student’s attempt to mark more than one answer to a question is interpreted as a blank answer. The asterisk (*) next to the number indicates the correct answer to the question.

Item difficulty refers to the proportion of students in the entire class marking the question correctly. Thus, if a question has a difficulty level of 90, it would be considered easy since it was correctly answered by 90% of those tested. Conversely, a question yielding a difficulty index of 10 (10% of the students answered the question correctly) would be regarded as an extremely difficult question.

The column under the heading, “Percent” is the difficulty index of each question. This is true for both Type 1 and Type 2 item analyses.

Type 2 High-Low Item Analysis

Item analysis generally yields information about the power of each item for discrimination between good and poor students and the level of difficulty of each question.

A High-Low Type 2 item analysis is identified by the phrase, “TOP AND BOTTOM 27%”, located in the right-hand side near the top of the computer printout. The column under the heading, “percent”, is the percentage of the high and low criterion groups which answered the question correctly. This is also the item difficulty. The column under the heading, “high”, is the fraction of the high criterion group which answered the question correctly; the column under the heading, “low”, is the fraction of the low criterion group which answered the question correctly. To convert these two scores into percentages, multiply by 100. Thus, a high value of .543 indicates that 54.3% of the high criterion group answered the question correctly. The column under the heading, “index” is the discrimination index of the question.

The discrimination index is obtained by comparing the question responses of students in the high and low criterion groups. If the question is efficient, students who earn the highest scores should tend to answer it correctly; students who earn the lowest test scores should tend to mark it incorrectly. A perfectly valid question would be one that is answered correctly by all of the high-scoring students and incorrectly by all of the low-scoring students. The discrimination index for such a question would be 1.000. The discrimination index declines as the validity of the question decreases and reaches a minimum value of .000 when equal fractions of good and poor students respond correctly. Such items do not contribute to the efficiency of the test. Questions are occasionally found to have a negative discrimination index. This means that a higher fraction of poor students rather than good students answered the question correctly.

This situation is very undesirable, and often indicates a faulty question or an error in the test key.

When there are multiple forms of a test, both types of item analysis include all test forms. The question numbers correspond to those on the master test.

The correct answer to each question is indicated by an asterisk printed immediately following the response number.

A Type 2 item analysis determines the validity of the test. The only students included in this item analysis are those students in the high and low criterion groups. The optimal percentage of the class to be in each of these groups is 27%. The high criterion group, therefore, includes the top 27% of the class as determined by the scores on this test; the low criterion group includes the bottom 27% of the class. A separate discrimination index for each question will be calculated.

Student Listing: The student listing was designed to provide lists that could be posted by student name (in alphabetical order) and/or Banner number (numerical order, last four digits only). The first page has the student’s Banner ID, name, score, percent correct z-score, and class section. On the next page are the Banner number and the score that was obtained. This may be used to post scores by Banner number.

Score: The actual number of correct responses.

Percent: The percent of the actual number of correct responses.

Z-Score: The z-score for each student can be found on the student listings for grade posting (see Student Listing, above). The z-score tells how many standard deviations the student’s score was from the class mean. If the z-score is greater than zero, this implies the score was above the mean; a negative z-score implies the score was below the mean. The larger the z-score, the greater the difference between the student’s score and the test mean.

List of Students Who Left Questions Blank: If a student fails to answer a question or if multiple marks were made for a question and the scanner operator had to erase all answers, this will list any questions left blank. The student's name may not be listed if he/she came out on the Test Message as STUDENT NOT CURRENTLY ENROLLED IN COURSE.

Additional Output Available

Separate Section Listing: An alphabetic listing by section can be obtained by merely requesting a separate section listing when the test is submitted for scoring. The form and content of this listing will be identical to the listing described under Student Listing. There will be a separate alphabetic listing for each section in addition to the alphabetic listing of the entire class.

Individual Student Output: One page printout for each student which lists each question that a student answered incorrectly. The wrong answer with the corresponding correct answer is listed.

Note: The program defaults to the Banner student ID for identification in this area. If a Banner student ID or Unique ID was not provided or the Banner number/UniqueID is not in the Student Banner Table for this course, the name will not appear on the printout.

Test Scramble: List of the scramble(s) as it was entered into the test.

Computerized Grade Books: Test data will be available at https://www.admin.miamioh.edu/cfapps/test_score/. The faculty's UniqueID must be provided on the TEST SCORING REQUEST FORM.

When downloading a test, a data set is created for each option selected.

Each answer sheet successfully scored will generate a record in the following format:

Character Position: Description

1-9: Banner number
10: Form number (if scramble is used)
11-30: Name
31: Blank
32-34: Score
35-39: Percent
40-44: Z-score
45-47: Section

Data Set Name for the SCO Data

Dataset of student scores (SCO) is in text format.

Data Set Name for the RAW Data

RAW data will be received in text format. A copy of the individual responses is electronically downloaded into a specified file. Each has the following format:

Character Position: Description

1-8: User Id - 00000000 for KEY
9: Blank
10: Form number
11-160: Questions 1-150

Data Set Name for the CSV Data

Data includes student Last Name, First Name, Username (UniqueID), Raw Score, Percent Score, and Standard Score. CSV data can be sorted and filtered in spreadsheet software such as Excel.

Return to Contents »