Interpreting Course Evaluation Data

Course evaluations, or student evaluation of teaching (SETs), are a tool widely used to measure teaching effectiveness in higher education and compare it across different courses, teachers, departments, and the institution at large. Course evaluation results can and have been used in a variety of ways including but not limited to individual course improvement, curriculum development, pedagogical development and refinement, promotion and tenure decisions, general education assessment, and accreditation activities. 

Student ratings and comments in course evaluations provide a single measure in a constellation of measurements of teaching effectiveness in higher education. As noted by Stephen L. Benton (2018), "effective instructor evaluation is complex and requires the use of multiple measures - formal and informal, traditional and authentic - as part of a balanced evaluation system". Indeed, the student voice in course evaluations is an important piece of this balanced evaluation system alongside self and peer evaluations of teaching effectiveness. It is important to provide a means for students to have a voice and for faculty to remain accountable to their students. It is also important to contextualize end-of-course evaluations of teaching effectiveness. Student ratings and comments provide one source of data. Further, it is the students' perspective at one particular moment in time, at the end of the course. As with other types of data, contextualizing data gathered from students in course evaluations is the keystone for fairly assessing teaching practices at any institution.

In the areas below, we will outline factors to think about when interpreting student evaluation data. We also provide further reading and other relevant resources. We also strongly recommend that review and promotion committees value student ratings and feedback data in light of self-assessment and peer review; taken together, these three areas offer meaningful data and analysis for evaluation. Reviewing the instructor's interpretation of student data - along with the instructor's discussion of instructional contexts, innovations, and classroom evidence - reflects robust best practice.

BEST PRACTICES, ADDITIONAL INFORMATION, & RESOURCES

Overview

Student evaluation of teaching data should be analyzed by looking for patterns and themes in the data from both quantitative and qualitative responses. 

Questions for identifying patterns and themes in data:

  • What patterns, if any, are indicated in the numerical ratings? Ratings for some items may help to make sense of ratings for others.
  • What patterns or themes are indicated in qualitative comments? Themes in qualitative comments can be quantified to demonstrate the degree of student consensus on particular aspects of the course.
  • Are there patterns across question types? Quantitative ratings may reflect points raised in students' open-ended comments or vice versa. Both sets of questions may inform an instructor's own self-assessment, peer review, or other forms of data.

Questions for contextualizing student data:

  • What is the teaching context? Take into account course characteristics such as the size of the course, whether an instructor is co-teaching, and whether the instructor supervises TAs. How do these ratings compare to: The instructor’s other courses; ratings for courses with similar sizes, levels, or content; or other courses with similar backgrounds and preparation?
  • What changes have occurred over time? What has improved?
  • What ratings stand out to the instructor and why? Instructors have the most context for the course, including their aims in teaching. What ratings do instructors find most useful for their own self-assessment in both strengths and areas for growth, and why?
  • What information is available to help clarify specific issues? Qualitative data, particularly, may help provide a more nuanced understanding of the course context or a particular issue raised in other data under review.

Best Practices for Interpretation of Course Evaluation Data

  • See ratings data as suggestive and not definitive. Student evaluations should be considered in the context of other evaluation materials such as peer review and self-assessment.
  • Account for variables. Consider the number and percentage of students who provide data (e.g., the larger the N, and the greater the percentage of students who respond, the more reliable the data). Additionally, consider whether the instructor used best practices for administering evaluations (see above section for examples).
  • Seek evidence of responsiveness to course evaluation results. Look for examples of changes in the course as a result of student feedback in evaluation summaries and other materials as applicable.
  • Be comprehensive in reading instructor's evaluation materials. While focusing on overall summative ratings, look at the distribution of student responses rather than just the mean score. We also recommend focusing on general categories (such as "Strongly Agree" and "Agree") rather than minor point differentials. If an instructor has offered multiple versions of the same course, aggregate data across courses.

External Resources