Advertisement

Responses of schools to accountability systems using multiple measures: the case of New York City elementary schools

  • M. C. M. Ehren
  • T. Hatch
Article

Abstract

Many studies point to potential unintended consequences of accountability systems such as when schools narrow their teaching to fixate on tested subjects. As a result, some states and districts in the USA have complemented the federal test-based accountability system with additional measures of educational practices to hold schools accountable on multiple measures. To explore the consequences of such systems, this study focuses on the responses of nine elementary schools to a multiple-measure accountability system in New York City, including high-stakes tests and quality reviews. While some schools showed broader improvement efforts, results suggest the state test remains the dominant measure in driving responses of schools, and in some cases, the quality review further reinforces the schools' focus on the test.

Keywords

Quality reviews High-stakes testing Educational accountability 

Notes

Acknowledgments

The research reported here was supported by the Spencer Foundation (grant agreement number 201100124).

References

  1. Baker, E. L. (2003). Multiple measures: toward tiered systems. Educational Measurement: Issues and Practice, 22(2), 13–17.CrossRefGoogle Scholar
  2. Barber, M. (2004). The virtue of accountability: system redesign, inspection, and incentives in the era of informed professionalism. Journal of Education, 85(1), 7–38.Google Scholar
  3. Booher-Jennings, J. (2005). Below the bubble: ‘Educational Triage’ and the Texas accountability system. American Educational Research Journal, 42(2), 231–268.CrossRefGoogle Scholar
  4. Chapman, C. (2001). Changing classrooms through inspection. School Leadership and Management, 21(1), 59–73.CrossRefGoogle Scholar
  5. Cullen, J.B., & Reback, R. (2006). Tinkering toward accolades: school gaming under a performance accountability system. Cambridge: National Bureau of Economic Research. Working paper 12286. http://www.nber.org/papers/w12286. Accessed Feb 2010.
  6. De Wolf, I.F., & Janssens, F.J.G. (2005). Effects and side effects of inspections and accountability in education; an overview of empirical studies. http://www1.fee.uva.nl/scholar/wp/wp53-05.pdf. Accessed Nov 2009.
  7. Ebrahim, A. (2005). Accountability myopia: losing sight of organizational learning. Nonprofit and Voluntary Sector Quarterly, 34(1), 56–87.CrossRefGoogle Scholar
  8. Ehren, M. C. M. (2006). Toezicht en schoolverbetering. Delft: Uitgeverij Eburon.Google Scholar
  9. Ehren, M. C. M., & Visscher, A. J. (2008). The relationship between school inspections, school characteristics and school improvement. British Journal of Educational Studies, 56(2), 205–227.CrossRefGoogle Scholar
  10. Figlio, D.N. and Getzler, L.S. (2002). Accountability, ability and disability: gaming the system. NBER working paper 9307. http://www.nber.org/papers/w9307. Accessed Dec 2009.
  11. Gong, B., & Hill, R. (2001). Some considerations of multiple measures in assessment and school accountability. Presentation at the Seminar on Using Multiple Measures and Indicators to Judge Schools' Adequate Yearly Progress Under Title I (sponsored by CCSSO & US DOE), Washington, DC.Google Scholar
  12. Gribben, M.A., Campbell, H.L., Mathew, J. (2008). Are advanced students advancing? Examining achievement trends beyond proficiency. Paper presented at AERA 2008.Google Scholar
  13. Haladyna, T. M., Nolen, S. B., & Haas, N. S. (1991). Raising standardized achievement test scores and the origins of test score pollution. Educational Researcher, 20(5), 2–7.CrossRefGoogle Scholar
  14. Hamilton, L.S., Stecher, B.M., Klein, S.P. (Eds.). (2002). Making sense of test-based accountability in education. Santa Monica: Rand cooperation. http://www.rand.org/pubs/monograph_reports/MR1554/. Accessed Feb 2010.
  15. Honig, M. I., & Hatch, T. C. (2004). Crafting coherence: how schools strategically manage multiple, external demands. Educational Researcher, 33(8), 16–30.CrossRefGoogle Scholar
  16. Jacob, B. A. (2005). Accountability, incentives and behavior: the impact of high-stakes testing in the Chicago public schools. Journal of Public Economics, 89(5–6), 761–796.CrossRefGoogle Scholar
  17. Jacob, B. A., & Levitt, S. D. (2003). Rotten apples: an investigation of the prevalence and predictors of teacher cheating. The Quarterly Journal of Economics (August), 118, 843–877.CrossRefGoogle Scholar
  18. Jos, P. H., & Tompkins, M. E. (2004). The accountability paradox in an age of reinvention; the perennial problem of preserving character and judgment. Administration and Society, 36(3), 255–281.CrossRefGoogle Scholar
  19. Kogan, M., & Maden, M. (1999). An evaluation of evaluators: the OFSTED system of school inspection. In C. Cullingford (Ed.), An inspector calls; Ofsted and its effect on school standards (9–32). London: Kogan Page Limited.Google Scholar
  20. Koretz, D. M. (2003). Using multiple measures to address perverse incentives and score inflation. Educational Measurement, 22(2), 18–26.CrossRefGoogle Scholar
  21. Koretz, D.M., McCaffrey, D.F., Hamilton, L.S. (2001). Towards a framework for validating gains under high-stakes conditions. CRESST/Harvard Graduate School of Education: CSE Technical Report 551Google Scholar
  22. Ladd, H.F. (2007). Holding schools accountable revisited. 2007 Spencer Foundation Lecture in Education Policy and Management.Google Scholar
  23. Leithwood, K., & Earl, L. (2000). Educational accountability effects: an international perspective. Peabody Journal of Education, 75(4), 1–18.CrossRefGoogle Scholar
  24. Linn, R. L. (2005). Conflicting demands of no child left behind and state systems: mixed messages about school performance. Education Policy Analysis Archives, 13(33), 1–17.Google Scholar
  25. Mehrens, W. A., & Kaminski, J. (1989). Methods for improving standardized test scores: fruitful, fruitless, or fraudulent? Educational Measurement: Issues and practice, 8(1), 14–22.CrossRefGoogle Scholar
  26. Mintrop, H., & Sunderman, G. L. (2009). Predictable failure of federal sanctions-driven accountability for school improvement-and why we may retain it anyway. Educational Researcher, 38(5), 353–364.CrossRefGoogle Scholar
  27. New York City Department of Education. (2009). Progress reports for NY City public schools (2009). http://schools.nyc.gov/. Retrieved from http://schools.nyc.gov/Accountability/default.htm. Accessed Jan 2010.
  28. New York City Department of Education. (2010a). Educator guide to the New York City progress report. http://schools.nyc.gov/. Retrieved from http://schools.nyc.gov/Accountability/default.htm. Accessed Jan 2010.
  29. New York City Department of Education. (2010b). New York City Department of Education letter to principals on changes in accountability system (27 September 2010). http://schools.nyc.gov/. Retrieved from http://schools.nyc.gov/Accountability/default.htm. Accessed Jan 2010.
  30. New York City Department of Education. (2010c). Survey results to improve schools; worksheet for school leaders. http://schools.nyc.gov/. Retrieved from http://schools.nyc.gov/Accountability/default.htm. Accessed Jan 2010.
  31. New York City Department of Education. (2010d). Quality review overview; principals and reviewers guide to the quality review. http://schools.nyc.gov/. Retrieved from http://schools.nyc.gov/Accountability/default.htm. Accessed Jan 2010.
  32. New York City Department of Education. (2010e). Principals guide to the quality review. http://schools.nyc.gov/. Retrieved from http://schools.nyc.gov/Accountability/default.htm. Accessed Jan 2010.
  33. New York City Department of Education (2010f). Quality review overview. http://schools.nyc.gov/. Retrieved from http://schools.nyc.gov/Accountability/default.htm. Accessed Jan 2010.
  34. New York City Department of Education. (2010g). Quality reviewer handbook. Retrieved from http://schools.nyc.gov/Accountability/default.htm. Accessed Jan 2010.
  35. New York State Education Department. (2008). Information bulletin New York State accountability system February 2008. http://www.nysed.gov/. Retrieved December 2010. Accessed Jan 2010.
  36. New York State Education Department. (2010a). How No Child Left Behind (NCLB) accountability works in New York State. http://www.nysed.gov/. Retrieved December 2010. Accessed Jan 2010.
  37. New York State Education Department. (2010b). Accountability peer review, update May 2010. http://www.nysed.gov/. Retrieved December 2010. Accessed Jan 2010.
  38. New York State Education Department. (2010c). Executive summary differentiated accountability. http://www.nysed.gov/. Retrieved December 2010. Accessed Jan 2010.
  39. New York State Education Department. (2010d). Proposal submitted to the US Education Department by the New York State Education Department to incorporate measures of student longitudinal growth into determinations of school and district AYP. http://www.nysed.gov/. Retrieved December 2010. Accessed Jan 2010.
  40. OECD (2005). Modernising government: the way forward, Paris, chapters 2–3. Available at www.oecdbookshop.org.
  41. Pallas, A. M., & Jennings, J. L. (2009). ‘Progress’ reports. In D. Ravitch, D. Meier, D. Avitia, D. C. Bloomfield, J. F. Brennan, H. N. Dukes, L. Haimson, E. N. Horowitz, J. L. Jennings, S. Koss, M. McAdoo, U. Ofer, A. M. Pallas, S. Sanders, S. Stern, P. J. Sulivan, & A. Wolf (Eds.), NYC schools under Bloomberg and Klein: what parents, teachers and policymakers need to know. New York: Cass Size Matters.Google Scholar
  42. Popham, W. J. (1991). Appropriateness of teachers' test-preparation practices. Educational Measurement: Issues and Practice, 10(4), 12–15.CrossRefGoogle Scholar
  43. Ravitch, D., Meier, D., Avitia, D., Bloomfield, D. C., Brennan, J. F., Dukes, H. N., Haimson, L., Horowitz, E. M., Jennings, J. L., Koss, S., McAdoo, M., Ofer, U., Pallas, A. M., Sanders, S., Stern, S., Sulivan, P. J., & Wolf, A. (2009). NYC schools under Bloomberg and Klein: what parents, teachers and policymakers need to know. New York: Cass Size Matters.Google Scholar
  44. Rosenthal, L. (2004). Do school inspections improve school quality? Ofsted inspections and school examination results in the UK. Economics of Education Review, 23(2), 143–152.CrossRefGoogle Scholar
  45. Stecher, B. M. (2002). Consequences of large-scale, high-stakes testing on school and classroom practices. Tests and their use in test-based accountability systems. In L. S. Hamilton, B. M. Stecher, & S. P. Klein (Eds.), Making sense of test-based accountability in education. Santa Monica: Rand cooperation. http://www.rand.org/pubs/monograph_reports/MR1554/. Accessed Feb 2010.
  46. Tetlock, P. E., Skitka, L., & Boettger, R. (1989). Social and cognitive strategies for coping with accountability: conformity, complexity and bolstering. Journal of Personality and Social Psychology, 57(4), 632–640.CrossRefGoogle Scholar
  47. The Council of School Supervisors and Administrators (CSA) (2009). Oversight - Department of Education's Progress Reports. Presented to: New York City Council, Committee on Education http://www.csa-nyc.org/. Retrieved from http://www.csa-nyc.org/sites/default/files/city%20council%20progress%20reports%20draft%20testimony%2010-30-%202009final.pdf. Accessed Jan 2011.
  48. Visscher, A. J., & Coe, R. (Eds.). (2002). School improvement through performance feedback (pp. 41–75). Lisse: Swets & Zeitlinger.Google Scholar
  49. Volante, L. (2004). Teaching to the test: what every educator and policy-maker should know. Canadian Journal of Educational Administration and Policy, 35, 1–6.Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Institute of EducationUniversity of LondonLondonUK
  2. 2.National Center for Restructuring Education, Schools, & Teaching (NCREST), Teachers CollegeColumbia UniversityNew YorkUSA

Personalised recommendations