Diagnosing Preservice Teachers’ Understanding of Statistics and Probability: Developing a Test for Cognitive Assessment
This study investigates preservice middle school mathematics teachers’ understanding of statistics and probability and provides a cognitive diagnostic assessment of their strengths and weaknesses on these subjects. A statistical reasoning test that included 15 multiple-choice and 5 open-ended items was developed from the perspective of the log-linear cognitive diagnosis model, which is a general form of the cognitive diagnosis models. The statistical reasoning test was applied to 456 preservice teachers from 4 universities in 3 different regions of Turkey. The collected data were analyzed using the Mplus 6.12 software, and diagnostic feedback on the preservice teachers’ responses was provided based on the findings. The analysis suggested that although many preservice teachers were able to represent and interpret the given data, most experienced difficulty in drawing inferences about populations based on samples, selecting and using appropriate statistical methods, and understanding and applying the basic concepts of probability. In addition, preservice teachers had difficulty answering open-ended items. Implications for teaching are also discussed.
KeywordsCognitive assessment Diagnostic classification models Preservice teacher education Statistics and probability Test design
Parts of this study were presented at the 2017 European Educational Research Conference held in Copenhagen/Denmark.
This study was conducted in equal collaboration between the two authors.
The authors disclose receipt of the following financial support for the research, authorship, and publication of this study: This study was supported by the Ahi Evran University Scientific Research Projects Coordination Unit. Project Number EGT.A3.16.014.
Compliance with Ethical Standards
Conflict of Interest
The authors declare that there are no conflicts of interest.
- Ball, D. L., Lubienski, S. T., & Mewborn, D. S. (2001). Research on teaching mathematics: The unsolved problem of teachers’ mathematical knowledge. In V. Richardson (Ed.), Handbook of research on teaching (4th ed., pp. 433–456). Washington, DC: American Educational Research Association.Google Scholar
- Batanero, C., & Díaz, C. (2010). Training teachers to teach statistics: What can we learn from research? Statistique et Enseignement, 1(1), 5–20.Google Scholar
- Batanero, C., & Díaz, C. (2012). Training school teachers to teach probability: Reflections and challenges. Chilean Journal of Statistics, 3(1), 3–13.Google Scholar
- Batanero, C., Godino, J. D., & Roa, R. (2004). Training teachers to teach probability. Journal of Statistics Education, 1(1), 5–20. Retrieved February 3, 2018, from http://www.amstat.org/publications/jse/v12n1/batanero.html.
- Baumert, J., Kunter, M., Blum, W., Brunner, M., Voss, T., Jordan, A., ... Tsai, Y. (2010). Teachers’ mathematical knowledge, cognitive activation in the classroom, and student progress. American Educational Research Journal, 47(1), 133–180.Google Scholar
- Bradshaw, L. (2015). PARCC diagnostic assessments for mathematics comprehension: A diagnostic classification model approach. Paper presented at the Council of Chief State School Officers (CCSSO) 2015 National Conference on Student Assessment (NCSA) in San Diego, California.Google Scholar
- Brown, A., & Silver, E. (1989). Data organization and interpretation. In M. M. Lindquist (Ed.), Results from the fourth mathematics assessment of the national assessment of educational progress (pp. 28–34). Reston, VA: National Council of Teachers of Mathematics.Google Scholar
- Burgess, T. A. (2007). Investigating the nature of teacher knowledge needed and used in teaching statistics (Unpublished doctoral dissertation). Massey University, Auckland, New Zealand.Google Scholar
- Choi, K. M., Lee, Y. S., & Park, Y. S. (2015). What CDM can tell about what students have learned: An analysis of TIMSS eighth grade mathematics. Eurasia Journal of Mathematics, Science & Technology Education, 11(6), 1563–1577.Google Scholar
- Common Core State Standards Initiative. (2010). The common core state standards for mathematics. Washington, DC: Author.Google Scholar
- delMas, R., Garfield, J., & Ooms, A. (2005, July). Using assessment items to study students’ difficulty with reading and interpreting graphical representations of distributions. Paper presented at the Fourth Forum on Statistical Reasoning, Thinking, and Literacy (SRTL–4), Auckland, New Zealand.Google Scholar
- delMas, R., Garfield, J., Ooms, A., & Chance, B. (2007). Assessing students’ conceptual understanding after a first course in statistics. Statistics Education Research Journal, 6(2), 28–58.Google Scholar
- Dereli, A. (2009). The mistakes and misconceptions in probability of eighth grade students (Unpublished Master’s Thesis). Eskisehir Osmangazi University, Turkey.Google Scholar
- DiBello, L. V., Stout, W. F., & Roussos, L. A. (1995). Unified cognitive/psychometric diagnostic assessment likelihood–based classification techniques. In P. Nichols, S. Chipman, & R. Brennan (Eds.), Cognitively diagnostic assessment (pp. 361–390). Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
- Fennema, E., & Franke, M. L. (1992). Teachers’ knowledge and its impact. In D. A. Grouws (Ed.), Handbook of research on mathematics teaching and learning (pp. 147–164). New York, NY: Macmillan.Google Scholar
- Franklin, C., Kader, G., Mewborn, D. S., Moreno, J., Peck, R., Perry, M., & Scheaffer, R. (2007). Guidelines for assessment and instruction in statistics education (GAISE) report: A pre–K–12 curriculum framework. Alexandria, VA: American Statistical Association. Retrieved February 8, 2018, from http://www.amstat.org/asa/education/Guidelines-for-Assessment-and-Instruction-in-Statistics-Education-Reports.aspx.
- Franklin, C., & Mewborn, D. (2006). The statistical education of PreK–12 teachers: A shared responsibility. In G. Burrill (Ed.), NCTM 2006 Yearbook: Thinking and reasoning with data and chance (pp. 335–344). Reston, VA: NCTM.Google Scholar
- Garfield, J., & Ben–Zvi, D. (2008). Developing students’ statistical reasoning: Connecting research and teaching practice. Berlin, Germany: Springer Science & Business Media.Google Scholar
- Garfield, J., & Ahlgren, A. (1988). Difficulties in learning basic concepts in probability and statistics: Implications for research. Journal for research in Mathematics Education, 19(1), 44–63.Google Scholar
- Groth, R. E. (2007). Toward a conceptualization of statistical knowledge for teaching. Journal for Research in Mathematics Education, 38(5), 427–437.Google Scholar
- Hartz, S. (2002). A Bayesian framework for the Unified Model for assessing cognitive abilities: Blending theory with practice (Unpublished doctoral dissertation). University of Illinois at Urbana–Champaign.Google Scholar
- Leavy, A. M. (2010). The challenge of preparing preservice teachers to teach informal inferential reasoning. Statistics Education Research Journal, 9(1), 46–67.Google Scholar
- Lee, H. S., & Hollebrands, K. F. (2011). Characterising and developing teachers’ knowledge for teaching statistics with technology. In C. Batanero, G. Burrill, & C. Reading (Eds.), Teaching statistics in school mathematics-challenges for teaching and teacher education (pp. 359–369). Dordrecht, Netherlands: Springer.Google Scholar
- Makar, K., & Rubin, A. (2009). A framework for thinking about informal statistical inference. Statistics Education Research Journal, 8(1), 82–105.Google Scholar
- Mathews, D., & Clark, J. (2003). Successful students’ conceptions of mean, standard deviation, and the central limit theorem. Unpublished paper. Retrieved July 4, 2018 from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.636.8870&rep=rep1&type=pdf.
- Muthen, L. K., & Muthen, B. O. (1998–2011). Mplus user’s guide (6th ed.). Los Angeles, CA: Muthen & Muthen.Google Scholar
- R Core Team. (2017). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved January 3, 2018, from http://www.R-project.org/.
- Ravand, H., & Robitzsch, A. (2015). Cognitive diagnostic modeling using R. Practical Assessment, Research & Evaluation, 20(11), 1–12.Google Scholar
- Rupp, A., Templin, J., & Henson, R. A. (2010). Diagnostic measurement: Theory, methods, and applications. New York, NY: Guilford Press.Google Scholar
- Sen, S., & Arican, M. (2015). A diagnostic comparison of Turkish and Korean students’ mathematics performances on the TIMSS 2011 assessment. Journal of Measurement and Evaluation in Education and Psychology, 6(2), 238–253.Google Scholar
- Shaughnessy, J. M. (2007). Research on statistics learning and reasoning. In F. K. Lester (Ed.), Second handbook of research on mathematics teaching and learning (pp. 957–1009). Reston, VA: The National Council of Teachers of Mathematics.Google Scholar
- Templin, J. (2008). Test construction item discrimination. Lecture presented at the Diagnostic Modelling Seminar at the University of Georgia, Athens. Retrieved February 2, 2018, from https://jonathantemplin.com/files/dcm/ersh9800f08/ersh9800f08_lecture11.pdf.
- Toker, T., & Green, K. (2012). An application of cognitive diagnostic assessment on TIMMS–2007 8th grade mathematics items. Paper presented at the annual meeting of the American Educational Research Association, Vancouver, British Columbia, Canada.Google Scholar
- von Davier, M. (2005). A general diagnostic model applied to language testing data. ETS Research Report. Princeton, NJ: Educational Testing Service.Google Scholar
- Watson, J. M. (2006). Statistical literacy at school: Growth and goals. Mahwah, NJ: Lawrence Erlbaum.Google Scholar
- Werner, C., & Schermelleh-Engel, K. (2010). Deciding between competing models: Chi–square difference tests. In Introduction to structural equation modeling with LISREL (pp. 1–3). Frankfurt, Germany: Goethe University.Google Scholar
- Zawojewski, J. S., & Heckman, D. S. (1997). What do students know about data analysis, statistics, and probability? In P. A. Kenny & E. A. Silver (Eds.), Results from the sixth mathematics assessment of the National Assessment of Educational Progress (pp. 195–223). Reston, VA: National Council of Teachers of Mathematics.Google Scholar
- Zawojewski, J. S., & Shaughnessy, J. M. (2000). Mean and median: Are they really so easy? Mathematics Teaching in the Middle School, 5(7), 436–440.Google Scholar