Reframing conversations about teacher quality: school and district administrators’ perceptions of the validity, reliability, and justifiability of a new teacher evaluation system
In changing accountability contexts, policymakers are engaging in international dialogue and collaborative efforts with new opportunities to reframe conversations about how to measure teacher quality and to (re) design and implement evaluation systems accordingly to ensure that they are fair, useable, feasible, and accurate. This study examined the lived experiences of school and district administrators in a large, fast-growth, suburban district in the USA regarding their districts’ new teacher evaluation system to better understand their perceptions of the system’s validity and reliability such that justifiable conclusions may be drawn about teachers’ effectiveness. Given concerns regarding validity and reliability, administrators generally discouraged external, high-stakes uses of evaluation results but valued the evaluation process and the data it provides for supporting teacher growth. As part of a larger study including teachers, findings can inform policymakers seeking to reform teacher evaluation frameworks to emphasize professional growth over high-stakes consequences.
KeywordsTeacher evaluation Accountability Educational policy Framework for personnel evaluation Administrator leadership Professional growth
- Amrein-Beardsley, A. (2014). Rethinking value-added models in education: critical perspectives on tests and assessment-based accountability. New York, NY: Routledge.Google Scholar
- Amrein-Beardsley, A., & Collins, C. (2012). The SAS education value-added assessment system (SAS® EVAAS®) in the Houston Independent School District (HISD): intended and unintended consequences. Education Policy Analysis Archives, 20. https://doi.org/10.14507/epaa.v20n12.2012.
- Baker, E. L., Barton, P. E., Darling-Hammond, L., Haertel, E., Ladd, H. F., Linn, R. L., et al. (2010). Problems with the use of student test scores to evaluate teachers. Washington, DC: Economic Policy Institute Retrieved from http://www.epi.org/publication/bp278/. Accessed 5 May 2013.Google Scholar
- Berliner, D. C. (2018). Between Scylla and Charybdis: reflections on and problems associated with the evaluation of teachers in an era of metrification. Education Policy Analysis Archives, 26(54). https://doi.org/10.14507/epaa.26.3820.
- Borg, S. (2018). Teacher evaluation: global perspectives and their implications for English language teaching. A literature review. London, UK: British Council Retrieved from https://www.teachingenglish.org.uk/sites/teacheng/files/pub_Teacher_evaluation_Global_perspectives_implications_ELT.pdf. Accessed 21 Dec 2018.Google Scholar
- Briggs, D., & Domingue, B. (2011). Due diligence and the evaluation of teachers: a review of the value-added analysis underlying the effectiveness rankings of Los Angeles Unified School District teachers by the Los Angeles Times. Boulder, CO: National Education Policy Center Retrieved from http://nepc.colorado.edu/publication/due-diligence. Accessed 4 Feb 2019.Google Scholar
- Centre for Development and Enterprise (CDE). (2015a). Teacher evaluation: lessons from other countries [number 8]. Johannesburg, South Africa: Author. Retrieved from https://www.cde.org.za/teacher-evaluation-lessons-from-other-countries/. Accessed 22 Dec 2018.
- Centre for Development and Enterprise (CDE). (2015b). Teacher evaluation in South African schools [number 9]. Johannesburg, South Africa: Author Retrieved from https://www.cde.org.za/teacher-evaluation-in-south-african-schools/. Accessed 22 Dec 2018.
- Collins, C., & Amrein-Beardsley, A. (2014). Putting growth and value-added models on the map: a national overview. Teachers College Record, 116(1), 1-34. Retrieved from http://www.tcrecord.org/Content.asp?ContentId=17291. Accessed 04 Feb 2019.
- Corcoran, S. P. (2010). Can teachers be evaluated by their students’ test scores? Should they be? The use of value-added measures of teacher effectiveness in policy and practice. Providence, RI: Annenberg Institute for School Reform Retrieved from http://www.annenberginstitute.org/publications/can-teachers-be-evaluated-their-students%E2%80%99-test-scores-should-they-be-use-value-added-me. Accessed 5 May 2013.Google Scholar
- Darling-Hammond, L. (2013). Getting teacher evaluation right: what really matters for effectiveness and improvement. New York, NY: Teachers College.Google Scholar
- Donaldson, M. L. (2011). Principals’ approaches to developing teacher quality: constraints and opportunities in hiring, assigning, evaluating, and developing teachers. Washington, DC: Center for American Progress Retrieved from http://cdn.americanprogress.org/wp-content/uploads/issues/2011/02/pdf/principal_report.pdf. Accessed 8 Aug 2013.Google Scholar
- Erickson, F. (1986). Qualitative methods in research on teaching. In M. C. Wittrock (Ed.), Handbook of research on teaching (pp. 119–161). New York, NY: Macmillan.Google Scholar
- Every Student Succeeds Act (ESSA) of 2015, Pub. L. No. 114-95, § 129 Stat. 1802. (2015).Google Scholar
- Fink, A. (1995). Evaluation for education and psychology. Thousand Oaks, CA: Sage.Google Scholar
- Finster, M., & Milanowski, A. (2018). Teacher perceptions of a new performance evaluation system and their influence on practice: a within- and between-school level analysis. Education Policy Analysis Archives, 26(41). https://doi.org/10.14507/epaa.26.3500.
- Glaser, B., & Strauss, A. (1967). The discovery of grounded theory: strategies for qualitative research. Chicago, IL: Aldine.Google Scholar
- Goldring, E., Grissom, J. A., Rubin, M., Neumerski, C. M., Cannata, M., Drake, T., & Schuermann, P. (2015). Make room value added: principals’ human capital decisions and the emergence of teacher observation data. Educational Researcher, 44(2), 96–104. https://doi.org/10.3102/0013189X15575031.Google Scholar
- Harris, D. N. (2011). Value-added measures in education: what every educator needs to know. Cambridge, MA: Harvard Education Press.Google Scholar
- Harris, D. N., Ingle, W. K., & Rutledge, S. A. (2014). How teacher evaluation methods matter for accountability: a comparative analysis of teacher effectiveness ratings by principals and teacher value-added measures. American Educational Journal, 51(1), 73–112. https://doi.org/10.3102/0002831213517130.Google Scholar
- Herlihy, C., Karger, E., Pollard, C., Hill, H. C., Kraft, M. A., Williams, M., & Howard, S. (2014). State and local efforts to investigate the validity and reliability of scores from teacher evaluation systems. Teachers College Record, 116, 1–28.Google Scholar
- Johnson, S. M. (2005). The prospects for teaching as a profession. In L. V. Hedges & B. Schneider (Eds.), The social organization of schooling (pp. 72–90). New York, NY: Russell Sage Foundation.Google Scholar
- Joint Committee on Standards for Educational Evaluation. (2008). The personnel evaluation standards (2nd ed.). Thousand Oaks, CA: Sage.Google Scholar
- Lavigne, A. L. (2014). Exploring the intended and unintended consequences of high-stakes teacher evaluation on schools, teachers, and students. Teachers College Record, 116, 1–29 Retrieved from http://www.tcrecord.org/library/abstract.asp?contentid=17294. Accessed 24 Jan 2015.Google Scholar
- Lavigne, A. L., & Good, T. L. (2014). Teacher and student evaluation: moving beyond the failure of school reform. New York, NY: Routledge.Google Scholar
- Loewus, L. (2017). Are states changing course on teacher evaluation? Test-score growth plays lesser role in six states. Education Week, 37(13), 1–17 Retrieved from https://www.edweek.org/ew/articles/2017/11/15/are-states-changing-course-on-teacher. Accessed 28 Feb 2018.Google Scholar
- Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis (2nd ed.). Thousand Oaks, CA: Sage Publications.Google Scholar
- No Child Left Behind (NCLB) Act of 2001, Public Law 107–110, § 115 Stat. 1425. (2002).Google Scholar
- Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York, NY: McGraw Hill.Google Scholar
- Organisation for Economic Co-operation and Development (OECD). (2013). Teachers for the 21st century: Using evaluation to improve teaching. Paris: OECD Publishing.Google Scholar
- Paufler, N. A. (2018a). Declining morale, diminishing autonomy, and decreasing value: principal reflections on a high-stakes teacher evaluation system. International Journal of Education Policy & Leadership, 13(8). https://doi.org/10.22230/ijepl.2018v13n8a831.
- Paufler, N. A. (2018b). The value a teacher evaluation system adds in practice: school administrator and teacher perceptions of system effectiveness. Manuscript submitted for publication.Google Scholar
- Popham, W. J. (1988). Educational evaluation (2nd ed.). Englewood Cliffs, NJ: Prentice Hall.Google Scholar
- Reddy, L. A., Dudek, C. M., Peters, S., Alperin, A., Kettler, R. J., & Kurz, A. (2018). Teachers’ and school administrators’ attitudes and beliefs of teacher evaluation: a preliminary investigation of high poverty school districts. Educational Assessment, Evaluation and Accountability, 30, 47–40. https://doi.org/10.1007/s11092-017-9263-3.Google Scholar
- Reid, D. B. (2018). School principals acting as middle leaders implementing new teacher evaluation systems. School Leadership & Management., 1–17. https://doi.org/10.1080/13632434.2018.1508013.
- Saldaña, J. (2013). The coding manual for qualitative researchers (2nd ed.). London: Sage.Google Scholar
- Stake, R. E., & Trumbull, D. (1982). Naturalistic generalizations. Review Journal of Philosophy and Social Science, 7, 1–12.Google Scholar
- Stewart, V. (2013). Teacher quality: the 2013 International Summit on the Teaching Profession. Asia Society: Partnership for Global Learning Retrieved from https://asiasociety.org/files/teachingsummit2013.pdf. Accessed 21 Dec 2018.Google Scholar
- Stewart, V. (2015). Implementing highly effective teacher policy and practice. Asia Society: Partnership for Global Learning Retrieved from https://asiasociety.org/files/2015-istp-report.pdf. Accessed 22 Dec 2018.Google Scholar
- Stewart, V. (2016). Teachers’ professional learning and growth: creating the conditions to achieve quality teaching for excellent learning outcomes. Asia Society: Partnership for Global Learning Retrieved from https://asiasociety.org/files/2016-istp-report-small.pdf. Accessed 22 Dec 2018.Google Scholar
- Stewart, V. (2018). New challenges and opportunities facing the teaching profession in public education. Asia Society: Partnership for Global Learning Retrieved from https://asiasociety.org/sites/default/files/inline-files/2018-international-summit-on-the-teaching-profession-edu-istp.pdf.Google Scholar
- Strauss, A. L., & Corbin, J. (1995). Basics of qualitative research: grounded theory procedures and techniques. Newbury Park, CA: Sage Publications.Google Scholar
- Taut, S., & Sun, Y. (2014). The development and implementation of a national, standards-based, multi-method teacher performance assessment system in Chile. Education Policy Analysis Archives, 22(71). https://doi.org/10.14507/epaa.v22n71.2014.
- Taylor, E. S., & Tyler, J. H. (2012). Can teacher evaluation improve teaching? Education Next, 12(4) Retrieved from https://www.educationnext.org/can-teacher-evaluation-improve-teaching/. Accessed 4 Feb 2019.
- Tucker, P. D., & Stronge, J. H. (2005). Linking teacher evaluation and student learning. Alexandria, VA: Association for Supervision and Curriculum Development.Google Scholar
- United Nations Educational, Scientific and Cultural Organization (UNESCO). (2016). Education 2030. Incheon declaration and framework for action. Retrieved from https://unesdoc.unesco.org/ark:/48223/pf0000245656. Accessed 22 Dec 2018.Google Scholar
- United States Department of Education. (2009). Race to the Top program: executive summary. Retrieved from http://www2.ed.gov/programs/racetothetop/executive-summary.pdf. Accessed 6 July 2013.
- United States Department of Education. (2010). Teacher Incentive Fund. Retrieved from http://www2.ed.gov/programs/teacherincentive/index.html. Accessed 6 July 2013.
- Weisberg, D., Sexton, S., Mulhern, J., & Keeling, D. (2009). The widget effect: our national failure to acknowledge and act of differences in teacher effectiveness (2nd ed.). Brooklyn, NY: The New Teacher Project (TNTP). Retrieved from http://tntp.org/ideas-and-innovations/view/the-widget-effect. Accessed 31 Jan 2018.
- Will, M. (2016). Assessing quality of teaching staff still complex despite ESSA’s leeway. Education Week, 36(16), 31–32 Retrieved from http://www.edweek.org/ew/articles/2017/01/04/assessing-quality-of-teaching-staff-still-complex.html?intc=EW-QC17-TOC&_ga=1.138540723.1051944855.1481128421. Accessed 9 Apr 2017.Google Scholar
- World Bank. (2013). What matters most in teacher policies? A framework for building a more effective teaching profession. Retrieved from https://openknowledge.worldbank.org/bitstream/handle/10986/20143/901820NWP0no4000Box385307B00PUBLIC0.pdf?sequence=1&isAllowed=y. Accessed 4 Feb 2019.