STUDENT EVALUATIONS OF TEACHING ARE MOSTLY AWFULLY WRONG
Noel Otu | Ntiense E Otu
Abstract:
Student evaluations of teaching (SETs) have been used, researched, and debated for many decades. It is a common practice in higher education institutions, with the supposed purpose of improving course quality and effectiveness, but with unintended consequences of encouraging and motivating poor teaching and causing grade inflation. There is strong evidence that SET “effectiveness” does not measure teaching effectiveness. This paper reviews empirical research examining common concerns about the usefulness (positive and negative) and accuracy of SETs. The findings reveal that student satisfaction relates to their anticipated/expected grades in their courses; hence, they want to get good grades and their instructors want to get a good rating of SET, and this results in grade inflation. The key points are that SETs (1) allow students to speak their “mind”, (2) have no compelling correlation between quality of teaching and learning effectiveness, (3) reward easy, less demanding, and lazy teachers with a positive rating, (4) are biased against gender, attractiveness, ethnicity, race, etc., (5) are weaponized against “some” faculty members, and (6) are like asking convicts awaiting sentencing to evaluate the judge or jurors who convicted them.
References:
- Allen, W. R., Epps, E. G., Guillory, E. A., Suh, S. A., & Bonous-Hammarth, M. (2000). The Black Academic: Faculty Status Among African Americans in U.S. Higher Education. Journal of Negro Education, 69(1-2), 112–127. Retrieved from https://psycnet.apa.org/record/2002-10183-007
- Accapadi, M.M. (2007) When White Women Cry: How White Women’s Tears Oppress Women of Color, The College Student Affair Journal 26 (2) p 208-215 (Google Scholar)
- Armstrong and Wildmen (2014) Working across Racial Lines in a Not-So-Post- Racial World, In Maritza Ryes, (Ed) Presumed Incompetent – The Intersections of Race and Class for Race and Class for Women in Academia Symposium, Plenary Panel, Retrieved from https://commons.law.famu.edu/cgi/viewcontent.cgi?article=1122&context=faculty-research
- Abel, R.L. (1990) Evaluating Evaluations: How should Law Schools Judge Teaching, 40J. Legal EDUC. 407-437. HEINOLIN Retrieved from https://heinonline.org/HOL/LandingPage?handle=hein.journals/jled40&div=40&id=&page=
- Aleamoni, L.M (1999) Student Rating Myths Versus Research Facts from 1924 to 1998 Journal of Personnel Evaluation in Education 13 no 2 p 153-166
- Anderson K.J. and Smith, G. (2005) Students’ Preconceptions of Professors: Benefit and Barriers According to Ethnicity and Gender, Hispanic Journal Behavioral Science, Vol. 27, issue 2.
- Boring, A (2017) Gender biases in student evaluations of teaching, Journal of Public Economics 145, 27-41 Retrieval from http://www.sciencedirect.com/science/article/pii/S0047272716301591 https://www.sciencedirect.com/science/article/abs/pii/S0047272716301591
- Boring, A., (2015). Gender Bias in Student Evaluations of Teachers, OFCE-PRESAGE- Sciences-Po Working Paper, Retrieved from http://www.ofce.sciences-po.fr/pdf/dtravail/WP2015-13.pdf
- Baker, P. and Copp, M., (1997) Gender Matters Most : The Interaction od Gendered Expectations, Feminist Course Content, And Pregnancy in Student Course Evaluations, Teaching Sociology , Vol 25, 29-43., Retrieved from https://www.jstor.org/stable/1319109?seq=1
- Benton, S.L.D, Dacon, D., and Pallett, W.H., (2013) Validity of Student Self- Reported Ratings of Learning “Assessment & Evaluation in Higher Education 3894 p. 377-388
- Beam, K. S. (1989) The Gender Gap in Law School Classroom Beyond Survival, 14 VT REV 23 29
- Borch, I., Sandvoilr and Risor, (2020) Discrepancies in Purposes of Student Course evaluations: What does it mean to be “satisfied”. Educational Assessment Evaluation and Accountability 3283-102 Retrieved from https://eric.ed.gov/?id=EJ1247575
- Bavistic, A., Madera, J.M., and Hebl, M.R. (2010) The Effect of Professor Ethnicity and Gender on Students Evaluations: Judged Before Met, Journal of Diversity in Higher Education 3, no 4: p 245-256
- Brooks, R.L., (1987) Anti-Minority Mindset in the Law School Personnel Process Towards an Understanding of Racial Mindset, 5 (1) Law & Inequality 1 Retrieved from https://scholarship.law.umn.edu/lawineq/vol5/iss1/1/
- Boex, L. (2000) Attributes of Effective Economics Instructors: An Analysis of Student Evaluations. The Journal of Economic Education, Vol. 32, No. 3, 211-227
- Burke, M.C., Hughey, A. W. and Cannonier, C., (2019) The Impact of Consumer and Academic Entitlement Attitudes on Student Course and Faculty Evaluations: A Call for Change to Academic Leaders, International Journal of Leadership and Change Vol 7 Issue I
- Czekanski, K., Hoerst, B. J., & Kurz, J. (2018). Instituting evidence-based changes to improve first-time NCLEX-RN® pass rates. Journal of Nursing Regulation, 9(1), 11-18. Retrieved from: https://doi.org/10.1016/S2155-8256(18)30049-8
- Chowning, K., and Campbell, N. J. (2009) Development and Validation of a Measure of Academic Entitlement: Individual Differences in Students' Externalized Responsibility and Entitled Expectations. Journal of Educational Psychology, 101 p 982-997
- Caron, P., (2018) Weaponizing Student Evaluation, Tax Prof Blog Blue Journal Retrieved from https://taxprof.typepad.com/taxprof_blog/2018/09/weaponizing-student-evaluations.html
- Clayson, D., (2009) Student Evaluations of Teaching: Are They Related to What Students Learn? Journal of Marketing Education 31(1) p16-30
- Cox, S., Rickard, M. K., and Lowery, C., (2021) The Student Evaluation of Teaching: Let’s be Honest – Who is Telling the Truth? Marketing Education Review. Vol. 32, Issue 1.
- Caldwell, K.J., (2018) Racial Bias and Student Ratings of Instructors, TEACHING SURVEY, Pitt: University Center for Teaching and Learning Retrieved from https://teaching.pitt.edu/wp-content/uploads/2018/12/OMET-Racial_bias_and_student_ratings_of_instructors.pdf
- Clayson, D. E., (2021), A Comprehensive Critique of Student Evaluation of Teaching: Critical Perspective on Validity, Reliability and Impartiality, Routledge, Taylor & Francis Group, eBook.
- Contra, J. (1993) Reflective Faculty Evaluation: Enhancing Teaching and Determining Faculty Effectiveness, (Jossey Bass higher and adult education series), San Francisco: Jossey Bass Inc Pub.
- Chavez, K., and Mitchell M.W. K (2019) Exploring Bias in Student Evaluations: Gender, Race, and Ethnicity, Cambridge University Press.
- Drowling, W. (2000) Why We Should Abolish Teaching Evaluations. The Daily Targum, Retrieved from www.wcdrutgers.net/targeval.htm
- Denzin, N., and Lincoln, Y. (2011) The Sage Handbook of Qualitative Research Association, Retrieved from: https://scholar.google.com/scholar?q=Denzin,+N.,+and+Lincoln,+Y+(2011)+The+sage+handbook+of+qualitative+research+Association&hl=en&as_sdt=0&as_vis=1&oi=scholart
- Denman, C. L., & Cohn, T. M. (2022). Use of Standardized Testing to Predict NCLEX-RN Success for Associate Degree Nursing Students in a Concept-Based Curriculum. Teaching and Learning in Nursing, 17(4), 378-382. Retrieved from: https://doi.org/10.1016/j.teln.2022.05.001
- Dabler-Larsen, P. (2009) Learning Oriented Educational Evaluation in Contemporary Society. In Ryan K. E.,(Ed), The Sage International Handbook of Educational Evaluations 307-322 Los Angeles, (Calif)
- Dano, T. and Stensaker, B., (2007) Still Balancing Improvement and Accountability! Development in External Quality Assurance in the Nordic Countries 1996-2006. Quality in Higher Education 13(1) 88-93.
- Dovidio, J.F., Gaertner, S.E., Kawakami, K. and Hodson, G (2002) Why Can’t We Just Get Along? Interpersonal Biases and Interracial Distrust. Cultural Diversity and Ethnic Minority Psychology Journal 8(2) Retrieved from https://psycnet.apa.org/record/2002-02415-002
- Edmunson, M. (1997) On the Use of Liberal Education as Lite Entitlement for Bored College Students, Harper's Magazine, 295. p 39-49. Retrieved from https://www.ljhammond.com/essay.htm
- Esarey. J and Valdes, N (2020) Unbiased, Reliable and Valid Student evaluations Can Still Be Unfair. Assessment & Evaluation in Higher Education, Vol 45, Issue 8, Retrieved from https://www.tandfonline.com/doi/full/10.1080/02602938.2020.1724875
- Feagin, J.R., and McKinney, K.D. (2003) The Many Cost of Racism, Rowman & Littlefield, Lanham, Md.
- Face, G.M.F (June 23, 2019) The Customer is Always Right is Awfully Wrong. The Canberra Times, Retrieved from https://www.canberratimes.com.au/story/6219583/heres-why-the-customer-is-always-right-is-awfully-wrong/
- Freishtat, R. L., (2016) Expert Report on Student Evaluations of Teaching, Ryerson Faculty Association and the Ontario Confederation of University Faculty, Retrieval from https://ocufa.on.ca/assets/RFA.v.Ryerson_Freishtat.Expert.Supplemental.Reports_2016.2018.pdf
- Fook, J., Ryan, M. and Hawking, L. (2000) Professional Expertise: Practice, Theory and Education for working in Uncertainty, London: Writing and Birch
- Frazier K.N. (2011) Academic Bullying: A Barrier to Tenure and Promotion for African A merican Faculty. Florida Journal of Educational Administration Policy, Fall Vol 5 Issue 1, p. 1-13
- Galbaith, C., Merrill, G., and Kline, D. (2012) Are Student Evaluations of Teaching Effectiveness Valid for Measuring Student Learning Outcomes in Business Related Classes? A Neutral Network and Bayesian Analyses. Research in Higher Education, Vol. 53(3) 353-374
- Hobson, S. M. and Donna, M.T. (2001) Understanding Student Evaluations: What All Faculty Should Know, College Teaching, Vol. 49, no 1. p 26-31
- Heath J.K., Weissman, G.E., Clancy, C.B., Shou, H., Farrar, J.T., & Dine, C.J. (2019) Assessment of Gender-Based Linguistic Differences in Physician Trainee Evaluations of Medical Faculty Using Automated Text Mining. JAMA Network Open, 2(5)
- Higgs, K., (2022) A Brief History of Consumer Culture, The MIT Press.
- Hefferman T. (2022) Abusive Comments in Student Evaluations of Courses and Teaching: The Attacks Women and Marginalized Academics Endure, High Education, Retrieved from https://link.springer.com/article/10.1007/s10734-022-00831-x
- Hamermesh D.S. and Parker, A (2005) Beauty in the Classroom Instructor’s Pulchritude and Putative Pedagogical Productivity, Economics of Education Review 24 (4) 369- 376.
- Himmeibach, V, (March 19,2019) Course Evaluation: What’s Wrong and How to Fix it., Retrieved from https://tophat.com/blog/course-evaluation/
- Hollis L D (2022) In the Room, But No Seat at the Table: Mixed Methods Analysis of HBCU Women Faculty and Workplace Bullying. Journal of education, Retrieved from https://journals.sagepub.com/doi/abs/10.1177/00220574221102329
- Huemer, M., (2001) Student Evaluation a Critical Review. PhilPaper, Retrieved from http://www.owl232.net/papers/sef.htm
- Islahi, F., and Nasreen, N. (2013) Who Make Effective Teachers, Men or Women? An Indian Perspective, Universal Journal of Educational Research, 1(4) p 285-293
- Johnson, V.E., (2003) Grade Inflation: A Crisis in College Education, New York, Springer
- Jan, T (April 7, 2010) Getting Students to Make Their Mark: Fewer Bothering to grade Professors as System Go Online. The Boston Globe, Boston.com.
- Kupenda in Reyes, M., (2014) Reflections on Presumed Incompetent The Intersections of Race and Class for Women in Academia. Symposium. The Plenary Panel, Scholary Commons @ FAMU from htttps://link.springer.com/article/10.1023/A:1026447822732
- Kopinak, J., (1999) The Use of Triangulation in a Study of Refugee Well Being, Quality and Quantity, International Journal of Methodology Springer, Vol. 33 (2) 169-183.
- Kafedric, E., Malec, D., and Niksic, E., (2018) Differences Between Male and Female Secondary School Students in Assessing Their Physical and Health Education Teachers’ Competence, Sport Science 11(1) p. 64-70
- Kreitzer, R. J., and Sweet-Cushman, J., (2022) Evaluating Student Evaluation of Teaching: A Review of Measurement and Equity Bias in SETS and Recommendations for Ethical Reform, Journal of Academic Ethics 20 (1) p. 73-84
- Lippmann, S., Bulanda, R.E, and Wagenaar, T. C. (2009) Student, Entitlement Question Issues and Strategies for Confronting Entitlement in the Classroom and Beyond. College Teaching, 57 p. 197-204.
- Li, D; Benton, S.L., Brown, R., Sullivan, P., and Ryalls, K. R;(2016) Analysis of idea Student Ratings of Instruction system 2015 Pilot Data. IDEA Technical Report No19 Assessed 1 October 2019, ERIC, Retrieved from https://eric.ed.gov/?id=ED573674
- Lim, M. H., Kwan, V.S.Y., Cheung, A., and Fiske, S.T. (2005) Stereotype Content Model Explains Prejudice for an Envied Outgroup: Scale Psychology Bulletin 31, no. 1, p 34-47
- Lawrence, J. W. (2021) Student Evaluations of teaching are not valid. American Association of University Professors. Retrieved from https://www.aaup.org/article/student-evaluations-teaching-are-not-valid#.Y_zgcXbMK70
- Laube, H., Massoni, K., Sprague, J., and Ferber, A. (2007) The Impact of Gender on the Evaluation of Teaching: What we know and what we can do. NWSA Journal 19(3), p 87-104
- M.A Martinez., Chang, A., and Welton, A.B. (2017) Assistant Professor of Color Confront the Inequitable Terrain of Academia? A Community Cultural Wealth Perspective, Race, Ethnicity and Education 20(5) p 696-710
- March, H. W., (1987) Student Evaluations of University Teaching: Research Findings, Methodological Issues and Directions for future research, International Journal of Educational Research, 11, 253-369.
- Mason A. D., (2010) Police Culture and Ethics Toward an Understanding and Expansion of Police Culture and Ethical Research, Graduate Theses and Dissertation paper 11482 Retrieved from https://dr.lib.iastate.edu/entities/publication/242a0143-5814-4e98-95e2-1a1df074de5d
- Merritt, D.J (2008) Bias, the Brain and Student Evaluations of Teaching, John, S’s Law Review, Volume 82 #1
- Mengel, F., Sauermann, J.,and Zolitz, U., (2019) Gender Bias in Teaching Evaluations, Journal of the European Economic Association, 17(2) p 535-566
- Mitchell, M.W., and Martin, J., (2018) Gender Bias in Student Evaluations: Political Science and Politics 51 (3)1-5
- Norris, D. (2019). Assessing the Learned Learner When Using a Concept Curriculum in Nursing Education. Research Issues in Contemporary Education, 4(n1) 35-44
- N/A #1 (2022) Course Feedback as a Measure of Teaching Effectiveness, Stanford Evaluation & Research, Retrieved from https://evals.stanford.edu/evaluating-teaching/course-feedback-measure-teaching-effectiveness
- Ortega - Luston, R., & Rodriguez - Soto, 1. (2014) Challenges, Choices and Decisions of Women in Higher Education: A Discourse on the future of Hispanic, Black, and Asian Members of the Professoriate, Journal of Hispanic Higher Education, 13 (4) p285-302
- Onueghergie, A., Daniel, E., and Collins, K. (2009) A Meta-Validation Model for Assessing the Score-Validity of Student-Teaching Evaluations. Quality and Quantity, 43(2), p. 197-209
- Olabi, R. M., (2021) The Pregnancy Drop: How Teaching Evaluations Penalize Pregnant Faculty, Humantities and Social Sciences Communications, Article number 253.
- Okoye, K., Arrona–Palarios, A., Camacho-Zuniga, C., Hammout N., Luttmann, E., Nakamura, J. E., and Hosseini, S., (2020) Impact of Students Evaluation of Teaching: A Text Analysis of the Teachers Qualities of Gender, International Journal of Educational Technology in Higher Education 17(49).
- Pounder, J. S., (2007) Is Student Evaluation of Teaching Worthwhile? An Analytical Framework for Answering the Question. Quality Assessment in Education Vol 15 No 2.
- Patitu, C., and Henton, K., (2003) The Experience of African American Women Faculty and Administrators in Higher Education: Has Anything Changed? New Direction for Student Services, 104. Wiley Periodical Online.
- Pope, D. C. (2001). “Doing School”: How We are Creating a Generation of Stressed Out, Materialistic, and Miseducated Students. New Haven, CT: Yale University Press
- Rodriguez, J., (2019) The Weaponization of Student Evaluations of Teaching: Bully and the Undermining of Academic Freedom, Journal of Academic Freedom, Vol 10
- Raban, C., (2007) Assurance “Versus” Enhancement: Less is More Journal of Further and Higher Education 31(1) 77-85. Retrieved from https://www.semanticscholar.org/paper/Assurance-versus-enhancement%3A-less-is-more-Raban/0ceb8a92d16aa073633b5aca4314227916a3a098
- Rempel, R. J., (2018) The Validity and Utility of Student Evaluation, Perspective on Social Work, Volume 13 no.1
- Reid, I.D., (2010) The Role of Perceived Race and Gender in the Evaluation of College Teaching on RateMyProfessor.com Journal of Diversity in Higher Education 3, no 3. P 137-274
- Reyes, M., Kupenda, A. M,, Onwuachi-Willig, A., Wildman, S. M., and Wing, A. K, (2014) Reflection on Presumed Incompetent: The Intersection of Race and Class for Women in Academia Symposium – The Plenary Panel, Scholarly Commons @ FAMU Law. Retrieved from http://digitalcommons.law.scu.edu/facpubs
- Radhakristnan, P., Frederickson, M., and Toh, S.M (2021) How Student Evaluations of teaching are Biased Against Women and Ethnic Minority Professors. ResearchGate. Retrieved from https://www.researchgate.net/publication/348152682_How_Student_Evaluations_of_Teaching_are_Biased_against_Women_and_Ethnic_Minority_Professors
- Ray, V (February 9, 2018) Is Gender Bias an Intended of Feature of Teaching Evaluation? Inside Higher ED.
- Rivera, L.A. and Tilesik, A. (2019) Scaling Down Inequality: Rating Scales, Gender Bias and the Architecture of Evaluation. American Sociological Review 84 (2) p 248-274
- Remmars, H. H. and Brandenburg, C. C. (1927) Experimental data on the Purdue Rating Scale for Instruction. Educational Administration and Supervision, 13, 519-527
- Remmer, H. H. (1933) Learning, Effort, and Attitudes as Affected by three methods of Instruction in Elementary Psychology, Purdue University Studies in Higher Education (Monograph No 21)
- Shulman, L (1988) Disciplines of Inquiry in Education: An Overview. In R. M. Jaeger (Ed) Complimentary Methods for Research in Education (p3-7) American Educational Research Association, Retrieved from https://www.aera.net/Publications/Books/Complementary-Methods-for-Research-in-Education
- Spooren, P., Brock, B. and Mortelman, D;(2013) On the Validity of Students Evaluation of Teaching: The state of the Art. Reviews of Educational Research, 83(4) p.598-642
- Seldin, P., (1984) Changing Practices in Faculty Evaluation, Jossey Bars, San Francisco, CA
- Seldin, P (1993) The Use and Abuse of Student Rating of Professors, The Chronicle of Higher Education, Vol 39 No 46. A40
- Searcy Y. D. (June 5, 2017) Let Me Speak to the Manager: Higher Education Should View the Public Not the Student as the Customer, INSIDE HIGHER EDUCATION
- Smith P.J. (1999) Teaching the Retrenchment Generation: When Sapphire Meet Sacrates at the Intersection of Race, Gender and Authority, 6 WM + Mary J. Women + L53, 162-163.
- Sieverding M, Eib. C., Neubauer A. B. + Stahl T., (2018) Can lifestyle preferences help explain the persistent gender gap in academia? The “mother work less” hypothesis supported for Germany but not for US early career researchers: PLOS ONE 13 (B): e0202728.
- Swann J.M. (2022) Academic Bullying and Diversity: Challenges and Solutions, The Scientist Forum, FEBS Letters/Volume 596, Issue 221p 2855-2858
- Spooren, P. Brockx, B., and Mortelmans, D. (2013), On the Validity of Student Evaluation of Teaching: The State of the Art. Review of Educational Research, 83(4), 598-642
- Sloman, S.A., (2002) Two Systems of Reasoning: in Gilovich, T and Kahnerman, D., (Ed) Heuristics and Biases: The Psychology of Intuitive Judgment, Camdridge University Press 279
- Stroebe, W, (2020) Student Evaluation of Teaching Encourages Poor Teaching and Contribute to Grade Inflation: A theoretical and Empirical Analysis, Basic and Applied Social Psychology Vol. 42Issue 4
- Stroebe, W, (2016) Why Good Teaching Evaluations May Reward Bad Teaching: On Grade Inflation and Other Unintended Consequences of Student Evaluation, Perspectives on Psychological Science, Vol 11(6) 800-816
- Sidanius, J and Crane, M (1989) Job Evaluation and Gender: The Case of University Faculty: Journal of Applied Social Psychology 19, no 2 p 174-197
- Subtirelu, N.C. (2015) She Does Have an Accent but – Race and Language Ideology in Student’s Evaluations of Mathematics Instructors on RateMyProfessors.com. Language in Society 44 no 1. p 35-62
- Smith, B.P. (2007) Student Ratings of Teaching Effectiveness: An Analysis of End of Course Faculty Evaluation, College Student Journal, 41, p 788-800
- Searcy Y.D. (2017) Let me speak to the manager! Higher Ed should redefine customer service, and craft policy with the public not the student most in mind, Inside Higher Ed Careers, Retrieved from https://www.insidehighered.com/views/2017/06/05/higher-ed-should-view-public-not-student-customer-essay
- Smith, W.A., Allen, W.R. and Danley, L.L., (2007) “Assume the position --- You fit the description, American Behavioral Scientist 51(4).
- Sabbe, E., + Aelterman, A (2007) Gender in Teaching: A literature Services. Teachers and Teaching, 13(5) p 521-538
- Stewart, K (2009) Lessons For Teaching Millennials, College Teaching 57p 111-118
- Theall M., and Franklin, J., (2000) Creating Responsive Student Rating System to Improve Evaluation Practice, New Directions for teaching and Learning 83, 95-107
- Tillman, L.C., (2001) Mentoring African American Faculty in Predominantly While Institutions, Research in Higher Education, 42(3) p295-325.
- Trower, C. A., and Chait, R. P., (2002) Faculty Diversity: Too Little for Too Long, Harvard Magazine, Retrieved from https://www.harvardmagazine.com/2002/03/faculty-diversity.html
- Titus, J. (2008) Student Ratings in Consumerist Academy: Leveraging Pedagogical Control and Authority. Sociological Perspectives 5/(2): 397-422.
- Uttl, B, and Smibert, D (May 9, 2017) Student Evaluation of Teaching: Teaching Quantitative Courses Can be Hazardous to One’s Career. Peer Journal, e3299.Retrieved from https://pubmed.ncbi.nlm.nih.gov/28503380/
- Williams, J., (2016) Quality Assurance and Quality Enhancement: is there a Relationship? Quality in Higher Education 22(2) 97-102.
- Ward D (2021) An overview of research on use of student surveys to evaluate teaching. University of Kansas,
- Wing, A. K (October 29, 2014) Lesson from a Portrait: Keep Calm and Carry On, in Reyes, Maritza I., Angela Mae Kupenda, Angela Onwuachi-Willig, Stephanie M. Wildman, Adrien Katherine Wing, Eds. Reflections on Presumed Incompetent: The Intersections of Race and Class for Women in Academia Symposium - The Plenary Panel, 29 Berkeley J. Gender L. & Just. 195 (2014)
- Wang, L and Gonzalez, J. A. (2020) Racial/Ethnic and National Origin Biases in SET. International Journal of Organizational Analysis. Vol 28 No 4