Abstract
Can we tell a good professor from their students’ comments? And are there differences between what is considered to be a good professor by different student groups? We use a large corpus of student evaluations collected from the RateMyProfessors website, covering different institutions, disciplines, and cultures, and perform several comparative experiments and analyses aimed to answer these two questions. Our results indicate that (1) we can reliably classify good professors from poor professors with an accuracy of over 90 %, and (2) we can separate the evaluations made for good professors by different groups with accuracies in the range of 71–89 %. Furthermore, a qualitative analysis performed using topic modeling highlights the aspects of interest for different student groups.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
The feature selection methods and the machine learning algorithms used in this study have been implemented in Python using the Sci-kit Learn machine learning library [16]. We use a maximum document frequency of 0.5 and lowercased text. We also experimented with stemming but it was found to degrade performance.
- 3.
In each of these figures, the topic distributions for a group add up to 100 % (e.g., the blue/dark and yellow/light columns in Fig. 3 each add up to 100 %).
References
Agarwal, A., Biadsy, F., Mckeown, K.R.: Contextual phrase-level polarity analysis using lexical affect scoring and syntactic n-grams. In: Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics, pp. 24–32. Association for Computational Linguistics (2009)
Bermingham, A., Smeaton, A.F.: Classifying sentiment in microblogs: is brevity an advantage? In: Proceedings of the 19th ACM International Conference on Information and Knowledge Management, pp. 1833–1836. ACM (2010)
Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)
Boyd, R.L., Wilson, S.R., Pennebaker, J.W., Kosinski, M., Stillwell, D.J., Mihalcea, R.: Values in words: using language to evaluate and understand personal values. In: Ninth International AAAI Conference on Web and Social Media (2015)
Brown, M.J., Baillie, M., Fraser, S.: Rating ratemyprofessors.com: a comparison of online and official student evaluations of teaching. Coll. Teach. 57(2), 89–92 (2009)
Coladarci, T., Kornfield, I.: Ratemyprofessors.com versus formal in-class student evaluations of teaching. Pract. Assess. Res. Eval. 12(6), 1–15 (2007)
Felton, J., Koper, P.T., Mitchell, J., Stinson, M.: Attractiveness, easiness and other issues: student evaluations of professors on ratemyprofessors.com. Assess. Eval. High. Educ. 33(1), 45–61 (2008)
Felton, J., Mitchell, J., Stinson, M.: Web-based student evaluations of professors: the relations between perceived quality, easiness and sexiness. Assess. Eval. High. Educ. 29(1), 91–108 (2004)
Freng, S., Webber, D.: Turning up the heat on online teaching evaluations: does hotness matter? Teach. Psychol. 36(3), 189–193 (2009)
Helterbran, V.R.: The ideal professor: student perceptions of effective instructor practices, attitudes, and skills. Education 129(1), 125 (2008)
Kim, S.M., Hovy, E.: Determining the sentiment of opinions. In: Proceedings of the 20th International Conference on Computational Linguistics, p. 1367. Association for Computational Linguistics (2004)
Lu, Y., Zhai, C.: Opinion integration through semi-supervised topic modeling. In: Proceedings of the 17th International Conference on World Wide Web, pp. 121–130. ACM (2008)
McCallum, A.K.: Mallet: a machine learning for language toolkit (2002). http://mallet.cs.umass.edu
Otto, J., Sanford, D.A., Ross, D.N.: Does ratemyprofessor.com really rate my professor? Assess. Eval. High. Educ. 33(4), 355–368 (2008)
Pang, B., Lee, L.: A sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts. In: Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, p. 271. Association for Computational Linguistics (2004)
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Stone, P., Earl, B.: A computer approach to content analysis: studies using the general inquirer system. In: Proceedings of the Spring Joint Computer Conference, ACM (1963)
Strapparava, C., Valitutti, A.: Wordnet affect: an affective extension of wordnet. In: Proceedings of the 4th International Conference on Language Resources and Evaluation, pp. 1083–1086 (2004)
Timmerman, T.: On the validity of ratemyprofessors.com. J. Educ. Bus. 84(1), 55–61 (2008)
Turney, P.D.: Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 417–424. Association for Computational Linguistics (2002)
Wilson, T., Wiebe, J., Hoffmann, P.: Recognizing contextual polarity in phrase-level sentiment analysis. In: Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, pp. 347–354. Association for Computational Linguistics (2005)
Acknowledgments
This material is based in part upon work supported by the National Science Foundation award #1344257 and by grant #48503 from the John Templeton Foundation. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation or the John Templeton Foundation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Azab, M., Mihalcea, R., Abernethy, J. (2016). Analysing RateMyProfessors Evaluations Across Institutions, Disciplines, and Cultures: The Tell-Tale Signs of a Good Professor. In: Spiro, E., Ahn, YY. (eds) Social Informatics. SocInfo 2016. Lecture Notes in Computer Science(), vol 10046. Springer, Cham. https://doi.org/10.1007/978-3-319-47880-7_27
Download citation
DOI: https://doi.org/10.1007/978-3-319-47880-7_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-47879-1
Online ISBN: 978-3-319-47880-7
eBook Packages: Computer ScienceComputer Science (R0)