Abstract
From a psychometric point of view, assessment means to infer what a learner knows and can do in the real world from limited evidence observed in a standardized testing situation. From a learning analytics perspective assessment means to observe real behavior in digital learning environments to conclude the learner status with the intent to positively influence the learning process. Although psychometrics and learning analytics share similar goals, for instance, formative assessment, while applying different methods and theories, the two disciplines are so far highly separated. This chapter aims at paving the way for an advanced understanding of assessment by comparing and integrating the learning analytics and the psychometric approach of assessment. We will discuss means to show this new way of assessment of educational concepts such as (meta-) cognition, motivation, and reading comprehension skills that can be addressed either from data-driven approach (learning analytics) or from a theory-driven approach (psychometrics). Finally, we show that radically new ways of assessment are located in the middle space where both disciplines are combined into a new research discipline called ‘Computational Psychometrics’.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aleven, V., Roll, I., McLaren, B. M., & Koedinger, K. R. (2010). Assessment of self-regulation during learning with an intelligent tutoring system. Educational Psychologist, 45, 224–233.
Anmarkrud, Ø., Bråten, I., & Strømsø, H. I. (2014). Multiple-documents literacy: Strategic processing, source awareness, and argumentation when reading multiple conflicting documents. Learning and Individual Differences, 30, 64–76. https://doi.org/10.1016/j.lindif.2013.01.007.
Arieli-Attali, M., Ward, S., Thomas, J., Deonovic, B., & von Davier, A. A. (2019). The expanded evidence-centered design (e-ECD) for learning and assessment systems: A framework for incorporating learning goals and processes within assessment design. Frontiers in psychology, 10(853). https://doi.org/10.3389/fpsyg.2019.00853.
Arnold, K., & Pistilli, M. D. (2012). Course signals at Purdue: using learning analytics to increase student success. In S. Buckingham Shum, D. Gasevic & R. Ferguson (Eds.), Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK 2012) (pp. 267–270). New York, NY, USA: ACM. http://dx.doi.org/10.1145/2330601.2330666.
Behrens, J. T., & DiCerbo, K. E. (2014). Harnessing the currents of the digital ocean. In J. Larussen & B. White (Eds.), Learning analytics: From research to practice (pp. 39–60). New York: Springer.
Biedermann, D., Schneider, J. & Drachsler, H. (2018). The learning analytics indicator repository. In 13th European Conference on Technology Enhanced Learning, EC-TEL 2018, Leeds, UK, September 3–5, 2018, Proceedings (Lecture Notes in Computer Science, Vol. 11082, pp. 579–582). Cham: Springer.
Blikstein, P. (2013). Multimodal learning analytics. In Proceedings of the Third International Conference on Learning Analytics and Knowledge—LAK 2013 (pp. 102–106). New York, USA: ACM. https://doi.org/10.1145/2460296.2460316.
Cukurova, M., Kent, C., & Luckin, R. (2019). The value of multimodal data in classification of social and emotional aspects of tutoring. AIED, 2(2019), 46–51.
DiCerbo, K. E., Shute, V., & Kim, Y. J. (2017). The future of assessment in technology rich environments: Psychometric considerations of ongoing assessment. In J. M. Spector, B. Lockee, & M. Childress (Eds.), Learning, design, and technology: An international compendium of theory, research, practice, and policy (pp. 1–21). New York, NY: Springer.
Di Mitri, D., Schneider, J., Specht, M., & Drachsler, H. (2018). From signals to knowledge: A conceptual model for multimodal learning analytics. Journal of Computer Assisted learning, 34, 338–349.
Di Mitri, D., Schneider, J., Specht, M., & Drachsler, H. (2019). Detecting mistakes in CPR training with multimodal data and neural networks. Sensors, 17, 3099.
Drachsler, H., & Schneider, J. (2018). Special Issue on multimodal learning analytics. Journal of Computer Assisted Learning, 34.
Echeverria, V., Martinez-Maldonado, R., Granda, R., Chiluiza, K., Conati, C., & Shum, S. B. (2018). Driving data storytelling from learning design. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge–LAK ’18 2018 (pp. 131–140). https://doi.org/10.1145/3170358.3170380.
Ferguson, R. (2012). Learning analytics: drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5/6), 304–317.
Goldhammer, F., & Zehner, F. (2017). What to make of and how to interpret process data. Measurement: Interdisciplinary Research and Perspectives, 15, 128–132.
Goldhammer, F., Hahnel, C., & Kroehne, U. (2020). Analyzing log file data from PIAAC. In D. B. Maehler & B. Rammstedt (Eds.), Large-scale cognitive assessment: Analyzing PIAAC data. Cham: Springer.
Gray, J., Chaudhuri, S., Bosworth, A., Layman, A., Reichart, D., Venkatrao, M., et al. (1997). Data cube: A relational aggregation operator generalizing group-by, cross-tab, and sub-totals. Data Mining and Knowledge Discovery, 1(1), 29–53.
Greller, W., & Drachsler, H. (2012). Translating learning into numbers. Journal of Educational Technology & Society, 15, 42–57.
Jivet, I., Scheffel, M., Drachsler, H., & Specht, M. (2017). Awareness is not enough. Pitfalls of learning analytics dashboards in the educational practice. In 12th European Conference on Technology-Enhanced Learning. Tallinn, Estonia, 12–15 September 2017.
Kroehne, U., & Goldhammer, F. (2018). How to conceptualize, represent, and analyze log data from technology-based assessments? A generic framework and an application to questionnaire items. Behaviormetrika, 45, 527–563.
Kosinski, M., Stillwell, D., & Graepel, T. (2013). Private traits and attributes are predictable from digital records of human behavior. Proceedings of the National Academy of Sciences, 110(15), 5802–5805. https://doi.org/10.1073/pnas.1218772110.
Mislevy, R. (2019). On integrating psychometrics and learning analytics in complex assessments. In H. Jiao, R. W. Lissitz, & A. van Wie (Eds.), Data analytics and psychometrics (pp. 1–52). Charlotte, NC, USA: Information Age Publishing.
Mislevy, R., Behrens, J., Dicerbo, K., & Levy, R. (2012). Design and Discovery in Educational Assessment: Evidence-Centered Design, Psychometrics, and Educational Data Mining. Journal of Educational Data Mining, 4(1), 11–48.
Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2003). A brief introduction to evidence-centered design. ETS Research Report Series, 2003(1), i–29. https://doi.org/10.1002/j.2333-8504.2003.tb01908.x.
Nguyen, Q., Huptych, M., & Rienties, B. (2018). Linking students’ timing of engagement to learning design and academic performance. Paper presented at the Proceedings of the 8th International Conference on Learning Analytics and Knowledge, Sydney, New South Wales, Australia.
Ochoa, X., & Worsley, M. (2016). Augmenting Learning Analytics with Multimodal Sensory Data. Journal of Learning Analytics, 3, 213–219. https://doi.org/10.18608/jla.2016.32.10.
OECD. (2013). OECD skills outlook 2013: First results from the survey of adult skills. Paris: OECD Publishing.
OECD. (2016). PISA 2015 results (Volume I): Excellence and equity in education. Paris: OECD Publishing.
Rienties, Bart, & Toetenel, Lisette. (2016). The impact of learning design on student behaviour, satisfaction and performance: a cross-institutional comparison across 151 modules. Computers in Human Behavior, 60, 333–341.
Scheffel, M., Drachsler, H., de Kraker, J., Kreijns, K., Slootmaker, A., & Specht, M. (2017). Widget, widget on the wall, am I performing well at all? IEEE Transactions on Learning Technologies, 10(1), 42–52. https://doi.org/10.1109/TLT.2016.2622268.
Scheffel, M., van Limbeek, E., Joppe, D., van Hooijdonk, J., Kockelkoren, C., Schmitz, M., Ebus, P., Sloep, P., & Drachsler, H. (2019). The means to a blend: A practical model for the redesign of face-to-face education to blended learning. In 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Delft, 16–19 September 2019, Proceedings (Lecture Notes in Computer Science). Cham: Springer.
Schön, D. (1983). The reflective practitioner. New York: Basic.
Schneider, J., Di Mitri, D., Limbu, B., & Drachsler, H. (2018). Multimodal learning hub: A tool for capturing customizable multimodal learning experiences. In 13th European Conference on Technology Enhanced Learning, EC-TEL 2018, Leeds, UK, 3–5, September 2018, Proceedings (Lecture Notes in Computer Science, Vol. 11082, pp. 45–58). Cham: Springer.
Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Dziurzynski, L., Ramones, S. M., Agrawal, M., et al. (2013). Personality, gender, and age in the language of social media: The open-vocabulary approach. PLoS ONE, 8(9), e73791. https://doi.org/10.1371/journal.pone.0073791.
Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189. https://doi.org/10.3102/0034654307313795.
Shute, V. J. (2011). Stealth assessment in computer-based games to support learning. Computer Games Instruction, 55, 503–524.
Tabuenca, B., Kalz, M., Drachsler, H., & Specht, M. (2015). Time will tell: The role of mobile learning analytics in self-regulated learning. Computers & Education, 89, 53–74.
Ternier S., Scheffel M., Drachsler H. (2018). Towards a cloud-based big data infrastructure for higher education institutions. In: Spector J. et al. (Eds), Frontiers of cyberlearning. Lecture Notes in Educational Technology. Springer, Singapore.
von Davier, A. A., Chung Wong, P., Polyak, S., & Yudelson, M. (2019). The argument for a “Data Cube” for large-scale psychometric data. Frontiers in Psychology, 4, 00071. https://doi.org/10.3389/feduc.2019.00071.
van der Linden, W. J., & Guo, F. (2008). Bayesian procedures for identifying aberrant response-time patterns in adaptive testing. Psychometrika, 73(3), 365–384.
von Davier, A. A. (2017). Computational psychometrics in support of collaborative educational assessments. Journal of Educational Measurement, 54(1), 3–11.
Weidenbach, M., Drachsler, H., Wild, F., Kreutter, S., Razek, V., Grunst, G., Ender, J., Berlage, T., & Janousek J. (2007). EchoComTEE—A simulator for transoesophageal echocardiography. Anaesthesia, 62(4), 347–357.
Winne, P. H. (2017). Leveraging big data to help each learner upgrade learning and accelerate learning science. Teachers College Record, 119(3).
Wise, S. L., & Kong, X. J. (2005). Response time effort: A new measure of examinee motivation in computer-based tests. Applied Measurement in Education, 18(2), 163–183.
Zieky, M. J. (2014). An introduction to the use of evidence-centered design in test development. Psicología Educativa, 20(2), 79–87. https://doi.org/10.1016/j.pse.2014.11.003.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Drachsler, H., Goldhammer, F. (2020). Learning Analytics and eAssessment—Towards Computational Psychometrics by Combining Psychometrics with Learning Analytics. In: Burgos, D. (eds) Radical Solutions and Learning Analytics. Lecture Notes in Educational Technology. Springer, Singapore. https://doi.org/10.1007/978-981-15-4526-9_5
Download citation
DOI: https://doi.org/10.1007/978-981-15-4526-9_5
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-4525-2
Online ISBN: 978-981-15-4526-9
eBook Packages: EducationEducation (R0)