Mathematical Problem Solving pp 209-229 | Cite as

# Beyond the Standardized Assessment of Mathematical Problem Solving Competencies: From Products to *Processes*

## Abstract

National and international standardised assessments of mathematical competencies challenge students to solve stimulating non-routine problems but, in some sense, they appear to have several limitations exactly in assessing students’ problem solving competencies. In particular, the use of multiple choice or short answer items focuses the assessment on the student’s choice (the product) rather than on how and why the student made that choice (the reasoning, that is, the process). The emersion of the processes appears to be a crucial need for teachers and researchers in order to understand what is involved in students’ answers. In this chapter, we will describe a protocol (developed within an Italian research project) for getting data to understand—at a micro level—the educational macro phenomena illustrated by standardised assessment quantitative results. This approach—that involves educational researchers and mathematics teachers—makes it possible to foster the development of a critical and productive approach to standardised tests, transforming them in an educational opportunity for teachers and researchers.

## References

- Bartolini Bussi, M. (1996). Mathematical discussion and perspective drawing in primary school.
*Educational Studies in Mathematics,**31,*11–41.CrossRefGoogle Scholar - Boaler, J. (2003). When learning no longer matters—Standardised testing and the creation of inequality.
*Phi Delta Kappan,**84*(7), 502–506.CrossRefGoogle Scholar - Bodin, A. (2005).
*What does PISA really assess? What it doesn’t? A French view*. Paper presented at the Joint Finnish-French Conference: Teaching Mathematics: Beyond the PISA Survey. Available to: www.apmep.fr/IMG/pdf/Com_PISA_FF_English.pdf. - Boero, P. (2011). Argumentation and proof: Discussing a “successful” classroom discussion. In M. Pytlak, T. Rowland, & E. Swoboda (Eds.),
*Proceedings of CERME-7*(pp. 120–130). Rszéskow, Poland.Google Scholar - Boero, P., Douek, N., & Ferrari, P. L. (2008). Developing mastery of natural language. Approaches to some theoretical aspects of mathematics. In L. English (Ed.),
*International handbook of research in mathematics education*(pp. 262–295). New York: Routledge.Google Scholar - Carvalho, L. M. (2012). The fabrications and travels of a knowledge-policy instrument.
*European Educational Research Journal,**11*(2), 172–188.CrossRefGoogle Scholar - Di Martino, P., & Baccaglini-Frank, A. (2017). Beyond performance results: Analyzing the informational and developmental potentials of standardized mathematics tests.
*For the Learning of Mathematics,**37*(3), 39–44.Google Scholar - Doig, B. (2006). Large-scale mathematics assessment: looking globally to act locally.
*Assessment in Education: Principles, Policy & Practice, 13*(3), 265–288.CrossRefGoogle Scholar - Duncker, K. (1945). On problem solving.
*Psychological Monographs, 58*(5), 1–113.CrossRefGoogle Scholar - Ekmekci, A., & Carmona, G. (2014). Studying mathematical literacy through the lens of PISA’s assessment framework. In C. Nicol, P. Liljedahl, S. Oesterle, & D. Allan (Eds.),
*Proceedings of the Joint Meeting of PME 38 and PME-NA 36*(Vol. 2, pp. 441–448). Vancouver, Canada.Google Scholar - Ferrer-Esteban, G. (2013).
*Rationale and incentives for cheating in the standardised tests of the Italian assessment system*. Programma Education FGA, Working Paper 50.Google Scholar - Grønmo, L., Lindquist, M., Arora, A., & Mullis, I. (2015).
*TIMSS 2015 mathematics framework*. Lynch School of Education, Boston University.Google Scholar - Halmos, P. (1980). The heart of mathematics.
*American Mathematical Monthly,**87*(7), 519–524.CrossRefGoogle Scholar - Jaworski, B., & Goodchild, S. (2006). Inquiry community in an activity theory frame. In J. Novotná, H. Moraová, M. Krátká, & N. Stehlíková (Eds.),
*Proceedings of the 30th Conference of the International Group for the Psychology of Mathematics Education*(Vol. 3, pp. 353–360). Prague, Czech Republic.Google Scholar - Kanes, C., Morgan, C., & Tsatsaroni, A. (2014). The PISA mathematics regime: Knowledge structures and practices of the self.
*Educational Studies in Mathematics,**87,*145–165.CrossRefGoogle Scholar - Liljedahl, P., Santos-Trigo, M., Malaspina, U., & Bruder, R. (2016).
*Problems solving in mathematics education*. ICME-13 Topical Surveys. Springer Open.Google Scholar - Mangez, E., & Hilgers, M. (2012). The field of knowledge and the policy field in education: PISA and the production of knowledge for policy.
*European Educational Research Journal,**11*(2), 189–205.CrossRefGoogle Scholar - Mayer, R. (1990). Problem solving. In M. Eysenck (Ed.),
*The Blackwell dictionary of cognitive psychology*(pp. 284–288). Oxford, UK: Basil Blackwell.Google Scholar - McLeod, D. (1989). Beliefs, Attitudes, and Emotions: New Views of Affect in Mathematics Education. In D. McLeod & V. Adams (Eds.),
*Affect and mathematical problem solving. A new perspective*(pp. 245–258). New York: Springer-Verlag.CrossRefGoogle Scholar - Mullis, I., Martin, M., Ruddock, G., O’Sullivan, C., Arora, A., & Erberber, E. (2005).
*TIMSS 2007 assessment frameworks*. MA: Chestnut Hill.Google Scholar - National Council of Teachers of Mathematics. (2000).
*Principles and standards for school mathematics*. VA: Reston.Google Scholar - Nevo, D. (2001). School evaluation: Internal or external?
*Studies in Educational Evaluation,**27*(2), 95–106.CrossRefGoogle Scholar - Niss, M. (2003). Mathematical competencies and the learning of mathematics: The Danish KOM project. In A. Gagatsis & S. Papastavridis (Eds.),
*Proceedings of the 3rd Mediterranean Conference on Mathematical Education*(pp. 115–124). Athens, Greece.Google Scholar - OECD. (2003).
*The PISA 2003 assessment framework—Mathematics, reading, science and problem solving knowledge and skills*. Paris: OECD Publishing.Google Scholar - OECD. (2006).
*Assessing scientific, reading and mathematical literacy: A framework for PISA 2006*. Paris: OECD Publishing.CrossRefGoogle Scholar - OECD. (2009).
*PISA 2009 assessment framework: Key competencies in reading, mathematics and science*. Paris: OECD Publishing.Google Scholar - OECD. (2013a).
*Synergies for better learning: An international perspective on evaluation and assessment, OECD reviews of evaluation and assessment in education*. Paris: OECD Publishing.CrossRefGoogle Scholar - OECD. (2013b).
*PISA 2012 assessment and analytical framework: Mathematics, reading, science, problem solving and financial literacy*. Paris: OECD Publishing.CrossRefGoogle Scholar - OECD. (2014).
*PISA 2012 results: Creative problem solving*(Vol. V). Paris: OECD Publishing.CrossRefGoogle Scholar - OECD. (2017).
*PISA 2015 draft collaborative problem solving framework*. Paris: OECD Publishing.Google Scholar - Papanastasiou, C. (2000). Internal and external factors affecting achievement in mathematics: Some findings from TIMSS.
*Studies in Educational Evaluation,**26,*1–7.CrossRefGoogle Scholar - Pellegrino, J. (2003). Knowing what students know.
*Issues in Science and Technology,**19*(2), 48–52.Google Scholar - Pons, X. (2012). Going beyond the ‘PISA shock’ discourse: An analysis of the cognitive reception of PISA in six European countries, 2001–2008.
*European Educational Research Journal,**11*(2), 206–226.CrossRefGoogle Scholar - Ribeiro, M., Mellone, M., & Jakobsen, A. (2016). Interpreting students’ non-standard reasoning: Insights for mathematics teacher education.
*For the Learning of Mathematics,**36*(2), 8–13.Google Scholar - Sowder, L. (1989). Searching for affect in the solution of story problems in mathematics. In McLeod & Adams (Eds.),
*Affect and mathematical problem solving: A new perspective*(pp. 104–113). New York: SpringerCrossRefGoogle Scholar - Verschaffel, L., Greer, B., & De Corte, E. (2000).
*Making sense of word problems*. The Netherlands: Swets & Zeitlinger.Google Scholar - Wijaya, A., Van den Heuvel-Panhuizen, M., Doorman, M., & Robitzsch, A. (2014). Difficulties in solving context-based PISA mathematics tasks: An analysis of students’ errors.
*The Mathematics Enthusiast,**11*(3), 555–584.Google Scholar - Wiliam, D. (2008). International comparisons and sensitivity to instruction.
*Assessment in Education: Principles, Policy and Practice, 15*(3), pp. 253–257.CrossRefGoogle Scholar - Zan, R. (2011). The crucial role of narrative thought in understanding story problems. In K. Kislenko (Ed.),
*Proceedings of the MAVI-16 Conference on Current State of Research on Mathematical Beliefs XVI*(pp. 287–305). Tallinn, Estonia.Google Scholar