Abstract
As educators begin to implement new curriculum standards like the Common Core State Standards and the Next Generation Science Standards, data from earlier reform efforts can provide critical information about what factors contribute to or impede the ability of new instructional materials to improve student learning. We used data from a National Science Foundation (NSF) funded Local Systemic Change (LSC) initiative to investigate how school principal and teacher buy-in impacted the effectiveness of two middle school curricula designed to implement the NCTM Standards. We found that, compared to matched Comparison schools, Treatment schools with the highest buy-in saw substantial gains in mathematics achievement, whereas Treatment schools with the lowest buy-in saw substantial declines in mathematics achievement.
Those who cannot remember the past are condemned to repeat it.
George Santayana
Research reported in this paper was supported by funding from the National Science Foundation Award Numbers: 9731483 and 0314806. Any opinions expressed herein are those of the authors and do not necessarily represent the views of the National Science Foundation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We attempted to keep raters as blind as possible to student test achievement data. Before they started their research, the qualitative researchers were instructed not to review such data. Further, we asked mentors not to discuss or compare standardized test data across districts when making their ratings. However, it is possible that during the time when mentors were working with the districts they had learned whether test scores were improving, and perhaps even developed some sense about which schools were seeing relatively weaker or relatively stronger improvement.
- 2.
- 3.
It might appear that, by assigning each Treatment school’s recentered Will-to-Reform score to its matched comparison schools, we are claiming that Will-to-Reform is a meaningful construct for the comparison schools, and further that the Will-to-Reform happens to be exactly the same at the matched comparisons as at the Treatment school. That is not what we have done. Will-to-Reform is our (retrospective and imperfect) measure of school-level buy-in at the Treatment school to their reform math curriculum. The matched comparison schools did not implement a reform math curriculum, so Will-to-Reform is not a meaningful concept for them. Within our HLM, by assigning the same value of Will-to-Reform to all members of a school-group we have made Will-to-Reform a variable that applies to school-groups, not to individual schools within a school-group. Conceptually, the HLM first estimates the growth over time at each school by computing slope for Year within that school. Then, the HLM estimates how Treatment affects the growth rate in each school-group by computing the slope for Treatment*Year within that school-group. Finally, the HLM estimates how Will-to-Reform impacts Treatment effects by computing across school groups the slope of Will-to-Reform*Treatment*Year. To be imprecise but conceptually correct, the model is treating Treatment*Year as a dependent variable with school-group as unit of analysis, and Will-to-Reform as the independent variable. In this way, parameter β9 in Eq. 2 estimates whether the effect of Treatment in school-groups where the Treatment school had a high Will-to-Reform is different from the effect of Treatment in school-groups where the Treatment school had a low Will-to-Reform.
References
Agodini, R., Harris, B., Thomas, M., Murphy, R., Gallagher, L., & Pendleton, A. (2010). Achievement effects of four elementary school math curricula: Findings for first and second graders. Washington, DC: Department of Education. Retrieved Jan 12, 2013, from http://ies.ed.gov/ncee/pubs/20094052/index.asp
Banilower, E. R., Boyd, S. E., Pasley, J. K., & Weiss, I. R. (2006). Lessons from a decade of mathematics and science reform: A capstone report for the local systemic change through teacher enhancement initiative. Chapel Hill, NC: Horizon Research, Inc.
Brown, A. L., & Campione, J. C. (1996). Psychological theory and the design of innovative learning environments: On procedures, principles, and systems. In R. Glaser (Ed.), Innovations in learning: New environments for education (pp. 289–325). Mahwah, NJ: Erlbaum.
Brown, M. W., & Edelson, D. C. (2001, April). Teaching by design: Curriculum design as a lens on instructional practice. Paper presented at the Annual Meeting of the American Educational Research Association, Seattle, WA.
Cai, J. (2003). What research tells us about teaching mathematics through problem solving. In F. Lester (Ed.), Research and issues in teaching mathematics through problem solving (pp. 241–254). Reston, VA: National Council of Teachers of Mathematics.
Cai, J. (2010). Evaluation of mathematics education programs. International Encyclopedia of Education, 3, 653–659.
Cai, J., Nie, B., & Moyer, J. C. (2010). The teaching of equation solving: Approaches in Standards-based and traditional curricula in the United States. Pedagogies: An International Journal, 5(3), 170–186.
Cai, J., Wang, N., Moyer, J. C., Wang, C., & Nie, B. (2011). Longitudinal investigation of the curriculum effect: An analysis of student learning outcomes from the LieCal Project. International Journal of Educational Research, 50(2), 117–136.
Cho, J. (1998, April). Rethinking curriculum implementation: Paradigms, models, and teachers’ work. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.
Confrey, J., Castillo-Chavez, C., Grouws, D., Mahoney, C., Saari, D., Schmidt, W., et al. (2004). On evaluating curricular effectiveness: Judging the quality of K-12 mathematics evaluations. Washington, DC: National Academies Press.
Council of Chief State School Officers and National Governors Association. (2010). Common core state standards for mathematics. Washington, DC: Council of Chief State School Officers and National Governors Association.
Council of Chief State School Officers, Brookhill Foundation, & Texas Instruments. (2011). Common Core State Standards (CCSS) mathematics curriculum materials analysis project. Washington, DC: Authors. Retrieved Jan 23, 2013, from https://www.k12.wa.us/CoreStandards/pubdocs/CCSSOMathAnalysisProj.pdf
Cross, C. T. (2004). Putting the pieces together: Lessons from comprehensive school reform research. Washington, DC: National Clearinghouse for Comprehensive School Reform.
Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8.
Dobson, L. D., & Cook, T. J. (1980). Avoiding type III error in program evaluation: Results from a field experiment. Evaluation and Program Planning, 3, 269–276.
Flay, B., Biglan, A., Boruch, R., Castro, F., Gottfredson, D., Kellam, S., et al. (2005). Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prevention Science, 6(3), 151–175.
Forgatch, M. S., Patterson, G. R., & DeGarmo, D. S. (2005). Evaluating fidelity: Predictive validity for a measure of competent adherence to the Oregon model of parent management training. Behavior Therapy, 36, 3–13.
Fullan, M., & Pomfret, A. (1977). Research on curriculum and instruction implementation. Review of Educational Research, 47, 335–397.
Garet, M. S., Cronen, S., Eaton, M., Kurki, A., Ludwig, M., Jones, W., et al. (2008). The impact of two professional development interventions on early reading instruction and achievement (NCEE 2008-4030). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.
Glennan, T. K., Bodilly, S. J., Galegher, J. R., & Kerr, K. A. (2004). Expanding the reach of education reform: Perspectives from leaders in the scale-up of educational interventions. Santa Monica, CA: Rand. Retrieved Jan 12, 2013, from http://www.rand.org/pubs/monographs/MG248.html
Goodlad, J. I. (1983). A study of schooling: Some implications for school improvement. Phi Delta Kappan, 64(8), 552–558.
Hiebert, J. (1999). Relationships between research and the NCTM Standards. Journal for Research in Mathematics Education, 30, 3–19.
Hohmann, A. A., & Shear, M. K. (2002). Community-based intervention research: Coping with the “noise” of real life in study design. American Journal of Psychiatry, 159, 201–207.
Kennedy, M. M. (2004). Reform ideals and teachers’ practical intentions. Education Policy Analysis Archives, 12(13). Retrieved Jan 2, 2013, from http://epaa.asu.edu/epaaa/v12n13/
Kilpatrick, J. (2003). What works? In S. L. Senk & D. R. Thompson (Eds.), NSF funded school mathematics curricula: What they are? What do students learn? (pp. 471–488). Mahwah, NJ: Erlbaum.
Krainer, K., & Peter-Koop, A. (2003). The role of the principal in mathematics teacher development. In A. Peter-Koop et al. (Eds.), Collaboration in teacher education (pp. 169–190). Dordrecht: Kluwer Academic.
Little, J. W. (1993). Teachers’ professional development in a climate of educational reform. Educational Evaluation and Policy Analysis, 15, 129–151.
National Council of Teachers of Mathematics. (1989). Curriculum and evaluation standards for school mathematics. Reston, VA: National Council of Teachers of Mathematics.
National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics. Reston, VA: The Council.
National Council of Teachers of Mathematics. (2009a). Curriculum focal points for prekindergarten through grade 8 mathematics: A quest for coherence. Reston, VA: The Council.
National Council of Teachers of Mathematics. (2009b). Focus in high school mathematics: fostering reasoning and sense making for all students. Reston, VA: The Council.
National Science Teachers Association. (2012). Recommendations on next generation science standards first public draft. Arlington, VA: NSTA. Retrieved Feb 6, 2013, from http://www.nsta.org/about/standardsupdate/recommendations2.aspx
New Jersey Department of Education. (1996). New Jersey core curriculum content standards for mathematics. Trenton, NJ: Author. Retrieved Aug 7, 2007, from http://www.edsolution.org/CustomizedProducts/data/standards-frameworks/standards/09mathintro.html
O’Donnell, C. L. (2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in KI-12 curriculum intervention research. Review of Educational Research, 78(1), 33–84.
O’Donnell, C. L. & Lynch, S. J. (2008, March). Fidelity of implementation to instructional strategies as a moderator of science curriculum unit effectiveness. Paper presented at the annual meeting of the American Educational Research Association, New York. Retrieved Jan 12, 2013, from http://www.gwu.edu/~scale-up/documents/AERA%20O%27Donnell%20Lynch%202008%20-%20Fidelity%20of%20Implementation%20as%20a%20Moderator.pdf
Pennsylvania Department of Education. (1999). Academic standards for mathematics. Harrisburg. PA: Author. Retrieved Aug 7, 2007, from http://www.pde.state.pa.us/k12/lib/k12/MathStan.doc
Remillard, J. T. (2005). Examining key concepts in research on teachers’ use of mathematics curricula. Review of Educational Research, 75(21), 1–246.
Riordan, J. E., & Noyce, P. E. (2001). The impact of two standards-based mathematics curricula on student achievement in Massachusetts. Journal for Research in Mathematics Education, 32(4), 368–398.
Romberg, T. A., Folgert, L., & Shafer, M. C. (2005).Differences in student performances for three treatment groups. (Mathematics in context longitudinal/cross-sectional study monograph 7). Madison, WI: University of Wisconsin, Wisconsin Center for Education Research. Retrieved May 28, 2014, from http://micimpact.wceruw.org/working_papers/Monograph%207%20Final.pdf
Rubin, D. (2001). Using propensity scores to help design observational studies: Application to the tobacco litigation. Health Services & Outcomes Research Methodology, 2, 169–188.
Schwartzbeck, T. D. (2002). Choosing a model and types of models: How to find what works for your school. Washington, DC: National Clearinghouse for Comprehensive School Reform.
Slavin, R. E. (2002). Evidence-based educational policies: Transforming educational practice and research. Educational Researcher, 31(7), 15–21.
Slavin, R. E., Lake, C. & Groff, C. (2008). Effective programs in middle and high school mathematics: A best-evidence synthesis. Baltimore, MD: Johns Hopkins University Center for Data Driven Reform in Education (CDDRE) Best Evidence Encyclopedia. Retrieved May 29, 2013, from http://www.bestevidence.org/word/mhs_math_Sep_8_2008.pdf
Stuart, E. A. (2007). Estimating causal effects using school-level data sets. Educational Researcher, 36, 187–198.
Tarr, J. E., Reys, R., Reys, B., Chávez, Ó., Shih, J., & Osterlind, S. (2008). The impact of middle school mathematics curricula on student achievement and the classroom learning environment. Journal for Research in Mathematics Education, 39(3), 247–280.
Turnbull, B. (2002). Teacher participation and buy-in: Implications for school reform initiatives. Learning Environments Research, 5(3), 235–252.
U.S. Department of Education. (2007a). What works clearinghouse intervention report: connected mathematics project. Washington, DC: Author. Retrieved Nov 1, 2007, from http://ies.ed.gov/ncee/wwc/pdf/WWC_CMP_040907.pdf
U.S. Department of Education. (2007b). Mathematics and science specific 84.305A RFA. Washington, DC: Author. Retrieved Dec 2, 2007, from http://ies.ed.gov/ncer/funding/math_science/index.asp
VanDerHeyden, A., McLaughlin, T., Algina, J., & Snyder, P. (2012). Randomized evaluation of a supplemental grade-wide mathematics intervention. American Educational Research Journal, 49(6), 1251–1284.
Vuchinich, S., Flay, B. R., Aber, L., & Bickman, L. (2012). Person mobility in the design and analysis of cluster-randomized cohort prevention trials. Prevention Science, 13(3), 300–313.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Kramer, S., Cai, J., Merlino, F.J. (2015). A Lesson for the Common Core Standards Era from the NCTM Standards Era: The Importance of Considering School-Level Buy-in When Implementing and Evaluating Standards-Based Instructional Materials. In: Middleton, J., Cai, J., Hwang, S. (eds) Large-Scale Studies in Mathematics Education. Research in Mathematics Education. Springer, Cham. https://doi.org/10.1007/978-3-319-07716-1_2
Download citation
DOI: https://doi.org/10.1007/978-3-319-07716-1_2
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-07715-4
Online ISBN: 978-3-319-07716-1
eBook Packages: Humanities, Social Sciences and LawEducation (R0)