Beyond the Standardized Assessment of Mathematical Problem Solving Competencies: From Products to Processes

  • Pietro Di MartinoEmail author
  • Giulia Signorini
Part of the ICME-13 Monographs book series (ICME13Mo)


National and international standardised assessments of mathematical competencies challenge students to solve stimulating non-routine problems but, in some sense, they appear to have several limitations exactly in assessing students’ problem solving competencies. In particular, the use of multiple choice or short answer items focuses the assessment on the student’s choice (the product) rather than on how and why the student made that choice (the reasoning, that is, the process). The emersion of the processes appears to be a crucial need for teachers and researchers in order to understand what is involved in students’ answers. In this chapter, we will describe a protocol (developed within an Italian research project) for getting data to understand—at a micro level—the educational macro phenomena illustrated by standardised assessment quantitative results. This approach—that involves educational researchers and mathematics teachers—makes it possible to foster the development of a critical and productive approach to standardised tests, transforming them in an educational opportunity for teachers and researchers.


  1. Bartolini Bussi, M. (1996). Mathematical discussion and perspective drawing in primary school. Educational Studies in Mathematics, 31, 11–41.CrossRefGoogle Scholar
  2. Boaler, J. (2003). When learning no longer matters—Standardised testing and the creation of inequality. Phi Delta Kappan, 84(7), 502–506.CrossRefGoogle Scholar
  3. Bodin, A. (2005). What does PISA really assess? What it doesn’t? A French view. Paper presented at the Joint Finnish-French Conference: Teaching Mathematics: Beyond the PISA Survey. Available to:
  4. Boero, P. (2011). Argumentation and proof: Discussing a “successful” classroom discussion. In M. Pytlak, T. Rowland, & E. Swoboda (Eds.), Proceedings of CERME-7 (pp. 120–130). Rszéskow, Poland.Google Scholar
  5. Boero, P., Douek, N., & Ferrari, P. L. (2008). Developing mastery of natural language. Approaches to some theoretical aspects of mathematics. In L. English (Ed.), International handbook of research in mathematics education (pp. 262–295). New York: Routledge.Google Scholar
  6. Carvalho, L. M. (2012). The fabrications and travels of a knowledge-policy instrument. European Educational Research Journal, 11(2), 172–188.CrossRefGoogle Scholar
  7. Di Martino, P., & Baccaglini-Frank, A. (2017). Beyond performance results: Analyzing the informational and developmental potentials of standardized mathematics tests. For the Learning of Mathematics, 37(3), 39–44.Google Scholar
  8. Doig, B. (2006). Large-scale mathematics assessment: looking globally to act locally. Assessment in Education: Principles, Policy & Practice, 13(3), 265–288.CrossRefGoogle Scholar
  9. Duncker, K. (1945). On problem solving. Psychological Monographs, 58(5), 1–113.CrossRefGoogle Scholar
  10. Ekmekci, A., & Carmona, G. (2014). Studying mathematical literacy through the lens of PISA’s assessment framework. In C. Nicol, P. Liljedahl, S. Oesterle, & D. Allan (Eds.), Proceedings of the Joint Meeting of PME 38 and PME-NA 36 (Vol. 2, pp. 441–448). Vancouver, Canada.Google Scholar
  11. Ferrer-Esteban, G. (2013). Rationale and incentives for cheating in the standardised tests of the Italian assessment system. Programma Education FGA, Working Paper 50.Google Scholar
  12. Grønmo, L., Lindquist, M., Arora, A., & Mullis, I. (2015). TIMSS 2015 mathematics framework. Lynch School of Education, Boston University.Google Scholar
  13. Halmos, P. (1980). The heart of mathematics. American Mathematical Monthly, 87(7), 519–524.CrossRefGoogle Scholar
  14. Jaworski, B., & Goodchild, S. (2006). Inquiry community in an activity theory frame. In J. Novotná, H. Moraová, M. Krátká, & N. Stehlíková (Eds.), Proceedings of the 30th Conference of the International Group for the Psychology of Mathematics Education (Vol. 3, pp. 353–360). Prague, Czech Republic.Google Scholar
  15. Kanes, C., Morgan, C., & Tsatsaroni, A. (2014). The PISA mathematics regime: Knowledge structures and practices of the self. Educational Studies in Mathematics, 87, 145–165.CrossRefGoogle Scholar
  16. Liljedahl, P., Santos-Trigo, M., Malaspina, U., & Bruder, R. (2016). Problems solving in mathematics education. ICME-13 Topical Surveys. Springer Open.Google Scholar
  17. Mangez, E., & Hilgers, M. (2012). The field of knowledge and the policy field in education: PISA and the production of knowledge for policy. European Educational Research Journal, 11(2), 189–205.CrossRefGoogle Scholar
  18. Mayer, R. (1990). Problem solving. In M. Eysenck (Ed.), The Blackwell dictionary of cognitive psychology (pp. 284–288). Oxford, UK: Basil Blackwell.Google Scholar
  19. McLeod, D. (1989). Beliefs, Attitudes, and Emotions: New Views of Affect in Mathematics Education. In D. McLeod & V. Adams (Eds.), Affect and mathematical problem solving. A new perspective (pp. 245–258). New York: Springer-Verlag.CrossRefGoogle Scholar
  20. Mullis, I., Martin, M., Ruddock, G., O’Sullivan, C., Arora, A., & Erberber, E. (2005). TIMSS 2007 assessment frameworks. MA: Chestnut Hill.Google Scholar
  21. National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics. VA: Reston.Google Scholar
  22. Nevo, D. (2001). School evaluation: Internal or external? Studies in Educational Evaluation, 27(2), 95–106.CrossRefGoogle Scholar
  23. Niss, M. (2003). Mathematical competencies and the learning of mathematics: The Danish KOM project. In A. Gagatsis & S. Papastavridis (Eds.), Proceedings of the 3rd Mediterranean Conference on Mathematical Education (pp. 115–124). Athens, Greece.Google Scholar
  24. OECD. (2003). The PISA 2003 assessment framework—Mathematics, reading, science and problem solving knowledge and skills. Paris: OECD Publishing.Google Scholar
  25. OECD. (2006). Assessing scientific, reading and mathematical literacy: A framework for PISA 2006. Paris: OECD Publishing.CrossRefGoogle Scholar
  26. OECD. (2009). PISA 2009 assessment framework: Key competencies in reading, mathematics and science. Paris: OECD Publishing.Google Scholar
  27. OECD. (2013a). Synergies for better learning: An international perspective on evaluation and assessment, OECD reviews of evaluation and assessment in education. Paris: OECD Publishing.CrossRefGoogle Scholar
  28. OECD. (2013b). PISA 2012 assessment and analytical framework: Mathematics, reading, science, problem solving and financial literacy. Paris: OECD Publishing.CrossRefGoogle Scholar
  29. OECD. (2014). PISA 2012 results: Creative problem solving (Vol. V). Paris: OECD Publishing.CrossRefGoogle Scholar
  30. OECD. (2017). PISA 2015 draft collaborative problem solving framework. Paris: OECD Publishing.Google Scholar
  31. Papanastasiou, C. (2000). Internal and external factors affecting achievement in mathematics: Some findings from TIMSS. Studies in Educational Evaluation, 26, 1–7.CrossRefGoogle Scholar
  32. Pellegrino, J. (2003). Knowing what students know. Issues in Science and Technology, 19(2), 48–52.Google Scholar
  33. Pons, X. (2012). Going beyond the ‘PISA shock’ discourse: An analysis of the cognitive reception of PISA in six European countries, 2001–2008. European Educational Research Journal, 11(2), 206–226.CrossRefGoogle Scholar
  34. Ribeiro, M., Mellone, M., & Jakobsen, A. (2016). Interpreting students’ non-standard reasoning: Insights for mathematics teacher education. For the Learning of Mathematics, 36(2), 8–13.Google Scholar
  35. Sowder, L. (1989). Searching for affect in the solution of story problems in mathematics. In McLeod & Adams (Eds.), Affect and mathematical problem solving: A new perspective (pp. 104–113). New York: SpringerCrossRefGoogle Scholar
  36. Verschaffel, L., Greer, B., & De Corte, E. (2000). Making sense of word problems. The Netherlands: Swets & Zeitlinger.Google Scholar
  37. Wijaya, A., Van den Heuvel-Panhuizen, M., Doorman, M., & Robitzsch, A. (2014). Difficulties in solving context-based PISA mathematics tasks: An analysis of students’ errors. The Mathematics Enthusiast, 11(3), 555–584.Google Scholar
  38. Wiliam, D. (2008). International comparisons and sensitivity to instruction. Assessment in Education: Principles, Policy and Practice, 15(3), pp. 253–257.CrossRefGoogle Scholar
  39. Zan, R. (2011). The crucial role of narrative thought in understanding story problems. In K. Kislenko (Ed.), Proceedings of the MAVI-16 Conference on Current State of Research on Mathematical Beliefs XVI (pp. 287–305). Tallinn, Estonia.Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of MathematicsUniversità di PisaPisaItaly

Personalised recommendations