Abstract
In large-scale studies of mathematics achievement, many important variables are at the classroom level and therefore require the collection of classroom observation data. However, in these studies, collecting observation data is expensive, and collecting numerous observations can be cost prohibitive. In addition to these practical considerations, there are methodological and conceptual considerations to measuring instruction. This chapter raises several of these issues around measuring mathematics classroom instruction by using empirical data from two prominent observational protocols.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Borko, H., Eisenhart, M., Underhill, R., Brown, C., Jones, D., & Agard, P. (1992). Learning to teach hard mathematics: Do novice teachers and their instructors give up too easily? Journal for Research in Mathematics Education, 23(3), 194–222.
Brennan, R. L. (2010). Generalizability theory. New York: Springer.
Chval, K. B., Reys, R. E., Reys, B. J., Tarr, J. E., & Chávez, O. (2006). Pressures to improve student performance: A context that both urges and impedes school-based research. Journal for Research in Mathematics Education, 37(3), 158–166.
Cohen, D., Raudenbush, S., & Ball, D. (2003). Resources, instruction, and research. Educational Evaluation and Policy Analysis, 25(2), 1–24.
Confrey, J., Battista, M. T., Boerst, T. A., King, K. D., Reed, J., Smith, M. S., et al. (2008). Situating research on curricular change. Journal for Research in Mathematics Education, 39(2), 102–112.
Correnti, R., & Martinez, J. F. (2012). Conceptual, methodological, and policy issues in the study of teaching: Implications for improving instructional practice at scale. Educational Assessment, 17, 51–61.
Hiebert, J. S., & Grouws, D. A. (2007). The effects of classroom mathematics teaching on students’ learning. In F. K. Lester Jr. (Ed.), Second handbook on mathematics learning and teaching (pp. 371–404). Charlotte, NC: Information Age Publishing.
Hill, H. C., Charalambos, C. Y., Blazar, D., McGinn, D., Kraft, M. A., Beisiegel, M., Litke, E., Lynch, K (2012). Validating arguments for observational instruments: Attending to multiple sources of variation. Educational Assessment, 17, 88–106.
Hill, H. C., Charalambous, C. Y., & Kraft, M. A. (2012). When rater reliability is not enough: Teacher observation systems and a case for the generalizability study. Educational Researcher, 41(2), 58–64.
Hill, H. C., Kapitula, L. R., & Umland, K. L. (2011). A validity argument approach to evaluating value-added scores. American Educational Research Journal, 48(3), 794–831.
Ing, M., & Webb, N. M. (2012). Characterizing mathematics classroom practice: Impact of observation and coding choices. Educational Measurement: Issues and Practice, 31(1), 14–26.
Ma, L. (1999). Knowing and teaching elementary mathematics: Teachers’ understanding of fundamental mathematics in China and the United States. Mahwah, NJ: Lawrence Erlbaum.
Marcoulides, G. A. (1993). Maximizing power in generalizability studies under budget constraints. Journal of Educational and Behavioral Statistics, 18(2), 197–206.
Marcoulides, G. A. (1997). Optimizing measurement designs with budget constraints: The variable cost case. Educational and Psychological Measurement, 57(5), 808–812.
Marcoulides, G. A., & Goldstein, Z. (1990). The optimization of generalizability studies with resource constraints. Educational and Psychological Measurement, 50(4), 761–768.
National Research Council. (2004). On evaluating curricular effectiveness: Judging the quality of K-12 evaluations. Washington, DC: National Academy Press.
Shavelson, R. J., & Webb, N. M. (1991). Generalizability theory: A primer. Newbury Park, CA: Sage.
Stigler, J. W., & Hiebert, J. (1999). The teaching gap: Best ideas from the world’s teachers for improving education in the classroom. New York: Free Press.
Tarr, J., Reys, R., Reys, B., Chávez, O., Shih, J., & Osterlind, S. (2008). The impact of middle grades mathematics curricula on student achievement and the classroom learning environment. Journal for Research in Mathematics Education, 39(3), 247–280.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Shih, J.C., Ing, M., Tarr, J.E. (2015). Addressing Measurement Issues in Two Large-Scale Mathematics Classroom Observation Protocols. In: Middleton, J., Cai, J., Hwang, S. (eds) Large-Scale Studies in Mathematics Education. Research in Mathematics Education. Springer, Cham. https://doi.org/10.1007/978-3-319-07716-1_16
Download citation
DOI: https://doi.org/10.1007/978-3-319-07716-1_16
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-07715-4
Online ISBN: 978-3-319-07716-1
eBook Packages: Humanities, Social Sciences and LawEducation (R0)