Abstract
We propose a probabilistic graphical model for predicting student attainment in web-based education. We empirically evaluate our model on a crowdsourced dataset with students and teachers; Teachers prepared lessons on various topics. Students read lessons by various teachers and then solved a multiple choice exam. Our model gets input data regarding past interactions between students and teachers and past student attainment. It then estimates abilities of students, competence of teachers and difficulty of questions, and predicts future student outcomes. We show that our model’s predictions are more accurate than heuristic approaches. We also show how demographic profiles and personality traits correlate with student performance in this task. Finally, given a limited pool of teachers, we propose an approach for using information from our model to maximize the number of students passing an exam of a given difficulty, by optimally assigning teachers to students. We evaluate the potential impact of our optimization approach using a simulation based on our dataset, showing an improvement in the overall performance.
Keywords
- Personality Trait
- Factor Graph
- Optimal Assignment
- Student Ability
- Probabilistic Graphical Model
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Chapter PDF
References
Anastasi, A., Urbina, S., et al.: Psychological testing. Prentice Hall, Upper Saddle River (1997)
Anderson, T.: The theory and practice of online learning. Au Press (2008)
Archak, N.: Money, glory and cheap talk: analyzing strategic behavior of contestants in simultaneous crowdsourcing contests on topcoder. com. In: Proceedings of the 19th International Conference on World Wide Web, pp. 21–30. ACM (2010)
Bachrach, Y., Graepel, T., Minka, T., Guiver, J.: How to grade a test without knowing the answers—a bayesian graphical model for adaptive crowdsourcing and aptitude testing. In: ICML (2012)
Bachrach, Y., Graepel, T., Kasneci, G., Kosinski, M., Van Gael, J.: Crowd iq: aggregating opinions to boost performance. In: AAMAS (2012)
Bachrach, Y., Graepel, T., Kohli, P., Kosinski, M., Stillwell, D.: Your digital image: factors behind demographic and psychometric predictions from social network profiles. In: AAMAS (2014)
Bachrach, Y., Kosinski, M., Graepel, T., Kohli, P., Stillwell, D.: Personality and patterns of facebook usage. In: ACM WebSci (2012)
Bachrach, Y., Rosenschein, J.S.: Distributed multiagent resource allocation in diminishing marginal return domains. In: AAMAS (2008)
Barrick, M.R., Mount, M.K.: The big five personality dimensions and job performance: a meta-analysis. Personnel Psychology 44(1), 1–26 (2006)
Brabazon, T.: Digital hemlock: Internet education and the poisoning of teaching. University of New South Wales Press (2002)
Brusilovsky, P., et al.: Adaptive and intelligent technologies for web-based eduction. KI 13(4), 19–25 (1999)
Clearwater, S.H.: Market-based control: A paradigm for distributed resource allocation (1996)
Costa Jr., P.T., McCrae, R.R.: Neo personality inventory–revised (neo-pi-r) and neo five-factor inventory (neo-ffi) professional manual. Psychological Assessment Resources, Odessa (1992)
Darling-Hammond, L.: Evaluating teacher effectiveness: How teacher performance assessments can measure and improve teaching (2010)
De Raad, B., Schouwenburg, H.C.: Personality in learning and education: A review. European Journal of Personality 10(5), 303–336 (1998)
Downes, S.: The rise of moocs. Stephens Web (2012)
Dumais, S.A.: Cultural capital, gender, and school success: The role of habitus. Sociology of Education, 44–68 (2002)
Eichler, H.-G., Kong, S.X., Gerth, W.C., Mavros, P., Jönsson, B.: Use of cost-effectiveness analysis in health-care resource allocation decision-making: How are cost-effectiveness thresholds expected to emerge? Value in health 7(5), 518–528 (2004)
Engle, J., Tinto, V.: Moving beyond access: College success for low-income, first-generation students. Pell Institute for the Study of Opportunity in Higher Education (2008)
Gao, X.A., Bachrach, Y., Key, P., Graepel, T.: Quality expectation-variance tradeoffs in crowdsourcing contests. In: AAAI (2012)
Getoor, L., Friedman, N., Koller, D., Pfeffer, A., Taskar, B.: 5 probabilistic relational models. Statistical Relational Learning, 129 (2007)
Gosling, S.D., Rentfrow, P.J., Swann, W.B.: A very brief measure of the big-five personality domains. Journal of Research in Personality 37(6), 504–528 (2003)
Glazerman, S., Loeb, S., Goldhaber, D., Staiger, D., Raudenbush, S., Whitehurst, G.: Evaluating teachers: The important role of value-added. Brookings Institution (2010)
Hambleton, R.K., Swaminathan, H., Rogers, H.J.: Fundamentals of item response theory, vol. 2 (1991)
Hara, N.: Student distress in a web-based distance education course. Information, Communication & Society 3(4), 557–579 (2000)
Harasim, L.: Shift happens: Online education as a new paradigm in learning. The Internet and Higher Education 3(1), 41–61 (2000)
Howe, J.: The rise of crowdsourcing. Wired Magazine 14(6), 1–4 (2006)
Khan, B.H.: Web-based instruction. Prentice Hall (1997)
Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques (2009)
Kosinski, M., Bachrach, Y., Kasneci, G., Van-Gael, J., Graepel, T.: Crowd iq: Measuring the intelligence of crowdsourcing platforms. In: ACM WebSci (2012)
Kosinski, M., Bachrach, Y., Kohli, P., Stillwell, D., Graepel, T.: Manifestations of user personality in website choice and behaviour on online social networks. Machine Learning 95(3), 357–380 (2014)
Kosinski, M., Stillwell, D., Graepel, T.: Private traits and attributes are predictable from digital records of human behavior. PNAS (2013)
Kosinski, M., Stillwell, D., Kohli, P., Bachrach, Y., Graepel, T.: Personality and website choice (2012)
Lavy, V.: Evaluating the effect of teachers group performance incentives on pupil achievement. Journal of Political Economy 110(6), 1286–1317 (2002)
Lev, O., Polukarov, M., Bachrach, Y., Rosenschein, J.S.: Mergers and collusion in all-pay auctions and crowdsourcing contests. In: AAMAS (2013)
Mackness, J., Mak, S., Williams, R.: The ideals and reality of participating in a MOOC. In: Networked Learing Conference (2010)
Minka, T., Winn, J.M., Guiver, J.P., Knowles, D.A.: Infer.NET 2.4 (2010)
Minka, T., Winn, J.: Gates. In: NIPS, vol. 21 (2008)
Minka, T.P.: A family of algorithms for approximate Bayesian inference. PhD thesis (2001)
Moore, M.G., Kearsley, G.: Distance education: A systems view of online learning. Wadsworth Publishing Company (2011)
Palloff, R.M., Pratt, K.: Lessons from the cyberspace classroom: The realities of online teaching. Wiley. Com (2002)
Pearl, J.: Probabilistic reasoning in intelligent systems: networks of plausible inference (1988)
Picciano, A.G.: Beyond student perceptions: Issues of interaction, presence, and performance in an online course. Journal of Asynchronous Learning Networks 6(1), 21–40 (2002)
Ridgell, S.D., Lounsbury, J.W.: Predicting academic success: General intelligence,big five personality traits, and work drive. College Student Journal 38(4), 607–618 (2004)
Russell, M.T., Karol, D.L.: Institute for Personality, and Ability Testing. In: The 16PF Fifth Edition Administrator’s Manual, Institute for Personality and Ability Testing, Champaign (1994)
Salek, M., Bachrach, Y., Key, P.: Hotspotting – a probabilistic graphical model for image object localization through crowdsourcing. In: AAAI (2013)
Schochet, P.Z., Chiang, H.S.: Error rates in measuring teacher and school performance based on student test score gains. ncee 2010-4004. National Center for Education Evaluation and Regional Assistance (2010)
Scott, J.: Family, gender, and educational attainment in britain: A longitudinal study. Journal of Comparative Family Studies 35(4), 565–590 (2004)
Sprinthall, R.C., Fisk, S.T.: Basic statistical analysis. Prentice Hall, Englewood Cliffs (1990)
Welinder, P., Branson, S., Belongie, S., Perona, P.: The multidimensional wisdom of crowds. In: NIPS (2010)
West, D.B., et al.: Introduction to graph theory, vol. 2. Prentice Hall, Upper Saddle River (2001)
Witkowski, J., Bachrach, Y., Key, P., Parkes, D.C.: Dwelling on the negative: Incentivizing effort in peer prediction. In: HCOMP (2013)
Yorke, M., Thomas, L.: Improving the retention of students from lower socio-economic groups. Journal of Higher Education Policy and Management 25(1), 63–74 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Shalem, B., Bachrach, Y., Guiver, J., Bishop, C.M. (2014). Students, Teachers, Exams and MOOCs: Predicting and Optimizing Attainment in Web-Based Education Using a Probabilistic Graphical Model. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2014. Lecture Notes in Computer Science(), vol 8726. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-44845-8_6
Download citation
DOI: https://doi.org/10.1007/978-3-662-44845-8_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-44844-1
Online ISBN: 978-3-662-44845-8
eBook Packages: Computer ScienceComputer Science (R0)