Understanding UK’s Teacher/Course Evaluation Reports

Each two-page TCE report shows the number and percentage of students responding to each of the questions, the average rating and, in the far right column, the standard deviation (labeled as SD). A standard deviation is shown only if 10 or more students responded. The number of students in the class (i.e., number of pre-slugged forms) as well as the number of completed forms returned (i.e., number of forms scanned) can be found at the bottom of page two.

Instructors are most familiar with the individual reports for their courses. However, other summary reports are available for university, college, department, and course prefix. Reports can also be generated by class size, course level, and instructor type for each department or college. The title on each report indicates how the data are summarized. Summary reports are distributed to colleges and departments shortly after results from the entire university are received. Online reports are available on the Office of Institutional Research, Planning, and Effectiveness’s Web Site

Guidelines for Interpreting the Evaluations

1. Use Multiple Sources of Information. It is critical to use other information about teaching in addition to student opinion. A great deal can be learned through an evaluation of instructional materials used in courses such as syllabi, texts, tests, homework assignments, availability of instructors to students for outside of classroom help, and advising practices. In addition, instructor self-reports, peer and administrator evaluations, and classroom visits can be used to learn about teaching performance. Student evaluation of teaching should not be used as the sole basis for determining teaching effectiveness.
2. Use Multiple Sets of Ratings. A pattern of ratings over time is the best estimate of instructor effectiveness as perceived by students. Ratings from only one course or from one term may not fairly represent an instructor’s performance. Some courses are not as highly rated as others. For personnel and merit review decisions, it is essential to examine rating patterns over time.
3. Obtain a Sufficient Number of Student Raters. Classes with less than five students should probably not participate in a standardized or objective evaluation of instruction. Resulting data are likely to be unreliable and invalid. Some researchers recommend 10 as the minimum student enrollment. For these very small classes, collecting information to open-ended questions and other comments from students will be more useful. The proportion of a class that rates an instructor also is important. If over one-third are absent or choose not to respond, the results may not be representative of the entire class.
4. Evaluate Statistics Carefully. Student ratings are often inflated. Institutional research at UK has shown that student ratings on the various four-point scales used in the evaluation generally exceed 2.5, the midpoint of the scale. For example, the overall mean on the item assessing the "overall quality of teaching" is roughly 3.4, with a standard deviation of approximately 0.8.
5. Consider the Impact of Course Characteristics. A few course characteristics appear to influence ratings and should be taken into account by reference to appropriate comparative data in other ways.
   • Small classes (with fewer than 15 students) often receive more favorable ratings than larger classes. However, very large classes are sometimes highly evaluated, possibly due to the fact that outstanding teachers are often assigned to these sections.
   • Upper level and graduate courses receive somewhat more favorable ratings than lower level courses. Courses are sometimes rated lower when they are perceived to be either too difficult or too elementary.
   • Required courses outside of a student’s major or minor field tend to receive somewhat lower ratings. Ratings also may differ because of the subject field of a course. For each of these characteristics, the differences may not be large, but together they can be significant.
   • Research has not substantiated that the time of day a course is offered, the term, the sex of the student, or the grade a student expects at the time of evaluation has any substantial effect on evaluations.
   For these reasons, it is often useful to compare student ratings of the same course taught by different instructors.
6. Interpret Comments Carefully. While written comments volunteered by students have a strong intuitive appeal to many instructors, content analysis research indicates that these remarks are not necessarily representative. In particular, comments on open-ended questions of a general nature cannot be generalized to the entire class. More specific questions that ask students to respond to a particular aspect of a course are usually more helpful to the instructor in improving the course or the instruction.
7. Evaluate Clarity of Supplemental Questions. Supplemental questions developed by the college, department, or instructor should be interpreted carefully. Examine whether the question is clearly worded and easily understood. If a question can be interpreted in various ways, poses more than one issue in a single statement, or is stated in such a way as to encourage one type of answer rather than another, students’ responses will not be meaningful.

Cashin, W. E. (1988). Student ratings of teaching: a summary of the research. Idea Paper No. 20. Manhattan, Kansas: Center for Faculty Evaluation and Development, Kansas State University.

Cashin, W. E. (1990). Student ratings of teaching: recommendations for use. Idea Paper No. 22. Manhattan, Kansas: Center for Faculty Evaluation and Development, Kansas State University.

Centra, J. A. (2003). Will teachers receive higher student evaluations by giving higher grades and less course work? Research in Higher Education, 44, 495-518.

Marsh, H. W. & Roche, L. A. (1997). "Making students’ evaluations of teaching effectiveness effective," American Psychologist, 52, 1187-97.