By LINDSAY ONUFER
Last month, with the support of 17 other scholarly societies, the American Sociological Association released a statement discouraging the use of student opinion of teaching surveys as the sole or primary means of measuring instructor teaching effectiveness, particularly for informing hiring, promotion, or tenure decisions. The statement’s authors argued that, while collecting student feedback is crucial in student-centered teaching practices, overreliance on teaching survey data is problematic, in part, because research indicates that survey responses can be influenced by student biases.
The significance of the effect of bias on student opinion of teaching surveys results varies across the research, but the evidence demonstrates that instructor identities like gender, race, and cultural background can affect teaching survey data (Fan et al., 2019; Mengel, Sauermann, & Zölitz, 2019; Mitchell & Martin, 2018; Smith & Hawkins, 2011).
The best way to mitigate negative effects caused by student bias in teaching survey results and to ensure equitable, comprehensive evaluation of teaching is to gather and examine evidence of teaching effectiveness from several different sources like reviews of course materials, peer observations, and instructor reflections, in addition to student feedback (American Sociological Association, 2019; Berk, 2005; Burdsal & Harrison, 2008). However, there are also several steps that instructors can take to reduce the effects of potential student biases and prepare students to deliver constructive feedback about their experiences in the course.
Align student-instructor course expectations
Early in the semester, talk with your students about what they expect and hope to learn in your course. In small and mid-sized classes, engage students in face-to-face whole class or small group discussions using your syllabus. In large classes, you can utilize surveys in Blackboard or Qualtrics or an audience response tool like Top Hat to collect information about student expectations. Use the discussion to explain course objectives, dispel misconceptions, address questions and concerns, and establish a shared understanding of the course and how it will progress. If you make changes to the course or syllabus throughout the semester, give students ample notice and explain your rationale. Telling students what to expect increases transparency, builds student-instructor rapport, and reduces the likelihood that students will perceive you or the course as disorganized.
Teach students how to deliver good feedback and model best practices
Research indicates that the most useful type of feedback is specific and improvement-focused (Hattie & Timperley, 2007). By demonstrating how to deliver good feedback, showing students examples of ineffective vs. effective feedback, building structured opportunities for student self and peer assessment into your course, and offering students targeted feedback, you can teach them how to progress from giving non-specific, unfocused positive or negative responses (e.g. “I liked/didn’t like this class.”) to specific, improvement-focused feedback that can be used to inform course or teaching improvements. In addition to improving the quality of feedback you receive on student opinion of teaching surveys, teaching students how to assess and give good feedback also has numerous academic benefits including improving students’ understanding of assessment and increasing academic performance and self-efficacy (Boud, 1995).
Gather student feedback in multiple ways at multiple times
Collecting student feedback throughout the term serves several purposes:
- It allows you to make improvements to the course during, as opposed to after, the semester.
- It allows you to use varied assessment methods like mid-semester surveys, classroom assessment techniques, or small group instructional diagnosis, a focus group-like protocol facilitated by a teaching consultant and used to collect more detailed, qualitative student feedback.
- It gives you a larger sample of student feedback and gives students multiple chances to offer feedback throughout the semester. This lets you to track students’ feedback over time and provides you with data that has been collected before the end of the semester when students’ responses might be influenced by stress or concerns about their final grades.
- Regardless of how or how often you gather student feedback, communicate to students how you utilize that data to make improvements to the course to convey that their feedback is used and useful.
Encourage students to complete student opinion of teaching surveys
Low student response rates can increase the impact of a small number of biased responses on overall teaching survey data. Telling students that you value and use their feedback, sending reminders to complete surveys, and giving students a small amount of class time to complete student opinion of teaching surveys can increase response rates.
Remind students of the purpose of student teaching surveys
Before students complete teaching surveys, explain that the purpose of the surveys is to gather their feedback about the course and their learning experience. Therefore, ratings or comments unrelated to teaching and learning (like feedback about when or where the course took place or about instructor characteristics other than teaching practices or instructional effectiveness) do not provide information that can be used to improve the course.
Talk to students about implicit bias
In a recent study, researchers found that adding language to student opinion of teaching surveys to increase students’ awareness of potential implicit bias prior to completing surveys resulted in less gender bias in students’ responses (Peterson, Biederman, Andersen, Ditonto, & Roe, 2019). The researchers noted that widespread adoption of such language at an institutional level might not produce the same results because students could become used to and ignore text that they saw every time they took a teaching survey. However, briefly explaining implicit bias and how it can influence survey results could help your students become aware of and manage their own unconscious biases while taking teaching surveys.
Need resources or assistance?
The Center for Teaching and Learning’s Assessment of Teaching Initiative assists schools and departments and individual instructors in planning assessment of teaching effectiveness. For additional information about support provided by the Teaching Center or more resources and readings, please visit the Assessment of Teaching Initiative website. To contact the Teaching Center for assistance, please email teaching@pitt.edu.
References
Berk, R.A. (2005). Survey of 12 strategies to measure teaching effectiveness. International Journal of Teaching and Learning in Higher Education, 17(1).
Boud, D. (1995). Enhancing learning through self-assessment. London: Kogan Page.
Burdsal, C.A. & Harrison, P.D. (2008). Further evidence supporting the validity of both a multidimensional profile and an overall evaluation of teaching effectiveness. Assessment & Evaluation in Higher Education, 33(5), 567-576. doi: 10.1080/02602930701699049
Fan, Y., Shepherd, L. J., Slavich, E., Waters, D., Stone, M., Abel, R., & Johnston, E. L. (2019). Gender and cultural bias in student evaluations: Why representation matters. PloS One, 14(2). doi: 10.1371/journal.pone.0209749
Hattie, J. and Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112. doi: 10.3102/003465430298487
Mengel, F., Sauermann, J., Zölitz, U. (2019). Gender bias in teaching evaluations. Journal of the European Economic Association, 17(2), 535-566. doi: 10.1093/jeea/jvx057
Mitchell, K. M. W., & Martin, J. (2018). Gender bias in student evaluations. PS: Political Science & Politics, 51(3), 648-652. doi: 10.1017/S104909651800001X
Peterson, D.A., Biederman, L. A., Andersen, D., Ditonto, T. M., & Roe, K. (2019). Mitigating gender bias in student evaluations of teaching. PloS One, 14(5). doi: 10.1371/journal.pone.0216241
Smith, B. P., & Hawkins, B. (2011). Examining student evaluations of black college faculty: Does race matter? The Journal of Negro Education, 80(2), 149–162.
Lindsay Onufer is the program manager of the Assessment of Teaching Initiative and a teaching consultant in the University Center for Teaching and Learning.