Abstract
This study investigates how uncertainty works in science policy debates by considering an unusual case: one in which uncertainty-based arguments for delay come from the scientific community, rather than industry actors. The case I present is the central use of value-added modeling (VAM) in the evaluation of individual teachers, a controversial trend in education reform. In order to understand how policy actors might counter inconvenient statements of uncertainty from experts, I analyze speeches from Education Secretary Arne Duncan, a committed and influential advocate of VAM. I identify a three-part rhetorical tactic, the “Overcaution Allegation,” and describe its persuasive potential to legitimize policies that elicit caution from the scientific community because they are built on uncertain science.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The legislation does not explicitly mention VAM, but instead uses terms like “measures of student growth.” In practice, these amount to value-added models or “student growth percentile” models (e.g., Goldhaber and Theobald 2012). In this study, it is unnecessary to differentiate between these or among differently specified value-added models. I will use the term VAM to encompass them all because their associated technical and methodological concerns are the same.
I focus on RTT in this paper because it is the Secretary’s most well-known initiative, but he pushed the same reform agenda (i.e., the central use of VAM to make high-stakes decisions about individual teachers) in his program for granting states waivers to the more onerous provisions of the No Child Left Behind Act.
I use this term as Collins and Evans (2002) do, to mean those actively involved in developing a body of knowledge (e.g., by experimenting, theorizing) or in resolving esoteric controversies should they arise. In the present case this means experts in quantitative, student-outcome-based approaches to the measurement of teacher practice.
References
AERA-American Educational Research Association. 2015. AERA Statement on use of value-added models (VAM) for the evaluation of educators and educator preparation programs. Educational Researcher 44(8): 448–452.
ASA-American Statistical Association. 2014. ASA statement on using value-added models for educational assessment. http://www.amstat.org/policy/pdfs/ASA_VAM_Statement.pdf. Accessed 2 Aug 2018.
Baker, Eva L., Paul E. Barton, Linda Darling-Hammond, Edward Haertel, Helen F. Ladd, Robert L. Linn, Diane Ravitch, Richard Rothstein, Richard J. Shavelson, and Lorrie A. Shepard. 2010. Problems with the use of student test scores to evaluate teachers. Washington, DC: Economic Policy Institute.
Banning, Marlia E. 2009. When poststructural theory and contemporary politics collide: The vexed case of global warming. Communication and Critical / Cultural Studies 6(3): 285–304.
Bohme, Susanna Rankin, John Zorabedian, and David S. Egilman. 2005. Maximizing profit and endangering health: Corporate strategies to avoid litigation and regulation. International Journal of Occupational and Environmental Health 11(4): 338–348.
Braun, Henry. 2015. The value in value added depends on the ecology. Educational Researcher 44(2): 127–131.
Ceccarelli, Leah. 2011. Manufactured scientific controversy: Science, rhetoric, and public debate. Rhetoric & Public Affairs 14(2): 195–228.
Collins, Harry M., and Robert Evans. 2002. The third wave of science studies: Studies of expertise and experience. Social Studies of Science 32(2): 235–296.
Compton, Mary, and Lois Weiner. 2008. The global assault on teaching, teachers, and their unions. New York: Palgrave MacMillan.
Corcoran, Sean. 2010. Can teachers be evaluated by their students’ test scores? Should they be? The use of value-added measures of teacher effectiveness in policy and practice. Providence, RI: Annenberg Institute for School Reform, Brown University. http://www.annenberginstitute.org/publications/can-teachers-be-evaluated-their-students%E2%80%99-test-scores-should-they-be-use-value-added-me. Accessed 2 Aug 2018.
Darling-Hammond, Linda, Audrey Amrein-Beardsley, Edward H. Haertel, and Jesse Rothstein. 2011. Getting teacher evaluation right: A background paper for policy makers. Washington, DC: American Educational Research Association and National Academy of Education.
Daston, Lorraine. 2000. The coming into being of scientific objects. In Biographies of scientific objects, ed. Lorraine Daston, 1–14. Chicago: University of Chicago Press.
DCPS-District of Columbia Public Schools. 2016. General education teachers with individual value-added student achievement data. http://dcps.dc.gov/sites/default/files/dc/sites/dcps/publication/attachments/1GeneralEducationTeacherswithIndividualValue-AddedStudentAchievementData.pdf. Accessed 2 Aug 2018.
Downer, John. 2014. Disowning Fukushima: Managing the credibility of nuclear reliability assessment in the wake of disaster. Regulation & Governance 8(3): 287–309.
Duncan, Arne. 2009a. Lawmakers who lead. U.S. Department of Education website. https://www.ed.gov/news/speeches/lawmakers-who-lead. Accessed 2 Aug 2018.
Duncan, Arne. 2009b. Make no small plans. U.S. Department of Education website. https://www.ed.gov/news/speeches/make-no-small-plans. Accessed 2 Aug 2018.
Duncan, Arne. 2009c. Partners in reform. U.S. Department of Education website. https://www.ed.gov/news/speeches/partners-reform. Accessed 2 Aug 2018.
Duncan, Arne. 2009d. Partners in truth-telling: Secretary Arne Duncan’s remarks to the Education Writers Association. U.S. Department of Education website. https://www.ed.gov/news/speeches/partners-truth-telling. Accessed 2 Aug 2018.
Duncan, Arne. 2009e. Reauthorization of ESEA: Why we can’t wait. U.S. Department of Education website. https://www.ed.gov/news/speeches/reauthorization-esea-why-we-cant-wait. Accessed 2 Aug 2018.
Duncan, Arne. 2009f. Robust data give us the roadmap to reform. U.S. Department of Education website. https://www.ed.gov/news/speeches/robust-data-gives-us-roadmap-reform. Accessed 2 Aug 2018.
Duncan, Arne. 2010. Education research: Charting the course for reform—Remarks by Secretary Arne Duncan at the Institute of Education Sciences research conference. U.S. Department of Education website. https://www.ed.gov/news/speeches/education-research-charting-course-reform-%E2%80%94-remarks-secretary-arne-duncan-institutes-education-research-conference. Accessed 2 Aug 2018.
Duncan, Arne. 2011a. A well-rounded curriculum in the age of accountability. U.S. Department of Education website. https://www.ed.gov/news/speeches/well-rounded-curriculum-age-accountability. Accessed 2 Aug 2018.
Duncan, Arne. 2011b. Lessons from high-performing countries. U.S. Department of Education website. https://www.ed.gov/news/speeches/lessons-high-performing-countries. Accessed 2 Aug 2018.
Duncan, Arne. 2011c. The road less traveled. U.S. Department of Education website. https://www.ed.gov/news/speeches/road-less-traveled. Accessed 2 Aug 2018.
Duncan, Arne. 2012a. Fighting the wrong education battles. U.S. Department of Education website. https://www.ed.gov/news/speeches/fighting-wrong-education-battles. Accessed 2 Aug 2018.
Duncan, Arne. 2012b. Remarks of U.S. Secretary of Education Arne Duncan at the Education Trust Conference. U.S. Department of Education website. https://www.ed.gov/news/speeches/remarks-us-secretary-education-arne-duncan-he-education-trust-conference. Accessed 2 Aug 2018.
Duncan, Arne. 2013a. Choosing the right battles: Remarks and a conversation. U.S. Department of Education website. https://www.ed.gov/news/speeches/choosing-right-battles-remarks-and-conversation. Accessed 2 Aug 2018.
Duncan, Arne. 2013b. The Tennessee way: Lessons for the nation. U.S. Department of Education website. https://www.ed.gov/news/speeches/tennessee-way-lessons-nation. Accessed 2 Aug 2018.
Duncan, Arne. 2015. Unleashing America’s energy for better education. U.S. Department of Education website. https://www.ed.gov/news/speeches/unleashing-americas-energy-better-education-legacy-race-top. Accessed 2 Aug 2018.
Fisher, Elizabeth, Judith Jones, and Rene von Schomberg (eds.). 2006. Implementing the Precautionary Principle: Perspectives and Prospects. Northampton, MA: Edward Elgar Publishing Inc.
Flechas, Joey. 2012. “Teachers upset with newly released appraisals.” The Gainesville Sun. December 4. http://www.gainesville.com/article/LK/20121204/News/604158832/GS/. Accessed 2 Aug 2018.
Freudenburg, William R., Robert Gramling, and Debra J. Davidson. 2008. Scientific Certainty Argumentation Methods (SCAMs): Science and the politics of doubt. Sociological Inquiry 78(1): 2–38.
Freudenburg, William R., and Susan K. Pastor. 1992. Public responses to technological risks: Toward a sociological perspective. Sociological Quarterly 33(3): 389–412.
Funtowicz, Silvio, and Roger Strand. 2007. Models of science and policy. In Biosafety first: Holistic approaches to risk and uncertainty in genetic engineering and genetically modified organisms, eds. Terje Traavik, and Lim Li Ching, 263–278. Trondheim, Norway: Tapir Academic Publishers.
Gardiner, Stephen M. 2006. A core precautionary principle. Journal of Political Philosophy 14(1): 33–60.
Godard, Olivier. 2006. The precautionary principle and catastrophism on tenterhooks: Lessons from a constitutional reform in France. In Implementing the Precautionary Principle: Perspectives and Prospects, eds. Elizabeth Fisher, Judith Jones, and René von Schomberg, 63–87. Northampton, MA: Edward Elgar Publishing Inc.
Goldhaber, Dan. 2015. Exploring the potential of value-added performance measures to affect the quality of the teacher workforce. Educational Researcher 44(2): 87–95.
Goldhaber, Dan, and Roddy Theobald. 2012. Do different value-added models tell us the same things?. Stanford: Carnegie Foundation for the Advancement of Teaching.
Haertel, Edward H. 2013. Reliability and validity of inferences about teachers based on student test scores. Princeton, NJ: Education Testing Service.
Harris, Douglas N. 2011. Value-added measures in education. Cambridge, MA: Harvard Education Press.
Harris, Douglas N., and Caroline Herrington. 2015. Editors’ introduction: The use of teacher value-added measures in schools: New evidence, unanswered questions, and future prospects. Educational Researcher 44(2): 71–76.
Jasanoff, Sheila. 2004. The idiom of co-production. In States of knowledge: The co-production of science and social order, ed. Sheila Jasanoff, 1–12. New York: Routledge.
Lee, Jason, Alpa Sridharan, and Anne Sung. 2014. “HISD gets a failing grade on its teacher evaluations.” The Houston Chronicle. May 7. https://www.chron.com/opinion/outlook/article/Lee-Sridharan-Sung-HISD-gets-a-failing-grade-5460933.php. Accessed 2 Aug 2018.
Markowitz, Gerald, and David Rosner. 2013. Deceit and denial: The deadly politics of industrial pollution. Berkeley: University of California Press.
McCaffrey, Daniel F., J.R. Lockwood, Daniel Koretz, and Laura S. Hamilton. 2003. Evaluating value-added models for teacher accountability. Santa Monica: RAND Corporation.
McCaffrey, Daniel F., Tim R. Sass, J.R. Lockwood, and Kata Mihaly. 2009. The intertemporal variability of teacher effect estimates. Education Finance and Policy 4(4): 572–606.
McGarity, Thomas O., and Wendy Elizabeth Wagner. 2008. Bending science: How special interests corrupt public health research. Cambridge: Harvard University Press.
McGoey, Linsey (ed.). 2014. An introduction to the sociology of ignorance: Essays on the limits of knowing. London: Routledge.
McIntush, Holly G. 2000. Defining education: The rhetorical enactment of ideology in “A Nation at Risk”. Rhetoric & Public Affairs 3(3): 419–443.
Michaels, David (ed.). 2008. Doubt is their product: How industry’s assault on science threatens your health. Oxford: Oxford University Press.
Michaels, David, and Celeste Monforton. 2005. Manufacturing uncertainty: Contested science and the protection of the public’s health and environment. American Journal of Public Health 95(S1): S39–S48.
Montopoli, Brian. (2009, March 10). “Obama’s remarks on education.” CBS News. March 10. https://www.cbsnews.com/news/obamas-remarks-on-education/. Accessed 2 Aug 2018.
Murdock, Linda. 2016. “‘Value-added’ measurement has little value: Using these numbers negatively impacts real people in real schools” Food for Thought about Public Education (blog). January 5. http://educornercafe.blogspot.com/2016/01/value-added-measurement-has-little.html. Accessed 2 Aug 2018.
NCTQ–National Council on Teacher Quality. 2013. State of the states 2013– Connect the dots: Using evaluations of teacher effectiveness to inform policy and practice. http://www.nctq.org/dmsView/State_of_the_States_2013_Using_Teacher_Evaluations_NCTQ_Report. Accessed 2 Aug 2018.
NCTQ–National Council on Teacher Quality. 2015. State of the states 2015–Evaluating teaching, leading and learning. http://www.nctq.org/dmsStage/StateofStates2015. Accessed 2 Aug 2018.
National Research Council. 2009. Letter report to the U.S. Department of Education on the Race to the Top fund. Washington, DC: National Academies Press.
National Research Council. 2010. Getting value out of value-added. Washington, DC: National Academies Press.
Oreskes, Naomi. 2015. The fact of uncertainty, the uncertainty of facts and the cultural resonance of doubt. Philosophical Transactions of the Royal Society A. https://doi.org/10.1098/rsta.2014.0455.
Oreskes, Naomi, and Erik M. Conway. 2010. Merchants of doubt: How a handful of scientists obscured the truth on issues from tobacco smoke to global warming. New York: Bloomsbury Press.
Papay, John. 2011. Different tests, different answers: The stability of teacher value-added estimates across outcome measures. American Educational Research Journal 48(1): 163–193.
Paroske, Marcus. 2009. Deliberating international science policy controversies: Uncertainty and AIDS in South Africa. Quarterly Journal of Speech 95(2): 148–170.
Phi Delta Kappa International. 2015. The 47th annual PDK/Gallup poll of the public’s attitudes toward the public schools. http://pdkpoll2015.pdkintl.org/wp-content/uploads/2015/10/pdkpoll47_2015.pdf. Accessed 2 Aug 2018.
Phi Delta Kappa International. 2016. The 48th annual PDK/Gallup poll of the public’s attitudes toward the public schools. http://pdkpoll2015.pdkintl.org/wp-content/uploads/2016/08/pdkpoll48_2016.pdf. Accessed 2 Aug 2018.
Proctor, Robert N. 1999. The Nazi war on cancer. Princeton, NJ: Princeton University Press.
Proctor, Robert N. 2008. Agnotology: A missing term to describe the cultural production of ignorance (and its study). In The making and unmaking of ignorance, eds. Robert N. Proctor and Londa Schiebinger, 1–33. Stanford, CA: Stanford University Press.
Ravitch, Diane. 2012. “No student left untested.” The New York Review of Books. February 21. http://www.nybooks.com/daily/2012/02/21/no-student-left-untested/. Accessed 2 Aug 2018.
Ravitch, Diane. 2014. Reign of error: The hoax of the privatization movement and the danger to America’s public schools. New York: Knopf.
Schochet, Peter, and Hanley Chiang. 2010. Error rates in measuring teacher and school performance based on student test score gains. Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. https://ies.ed.gov/ncee/pubs/20104004/pdf/20104004.pdf. Accessed 2 Aug 2018.
Smithson, Michael. 1980. Interests and the growth of uncertainty. Journal for the Theory of Social Behavior 10(3): 157–168.
Stecher, Brian M., Deborah J. Holtzman, Michael S. Garet, Laura S. Hamilton, John Engberg, Elizabeth D. Steiner, Abby Robyn, Matthew D. Baird, Italo A. Gutierrez, Evan D. Peet, Iliana Brodziak de los Reyes, Kaitlin Fronberg, Gabriel Weinberger, Gerald Paul Hunter, and Jay Chambers. 2018. Improving teaching effectiveness: The intensive partnerships for effective teaching through 2015–2016. Santa Monica, CA: RAND Corporation. https://www.rand.org/pubs/research_reports/RR2242.html. Accessed 2 August 2018.
Steudeman, Michael J. 2014. “The guardian genius of democracy”: The myth of the heroic teacher in Lyndon B. Johnson’s education policy rhetoric, 1964–1966. Rhetoric & Public Affairs 17(3): 477–510.
Stocking, Holly, and Lisa W. Holstein. 2008. Manufacturing doubt: Journalists’ roles and the construction of ignorance in a scientific controversy. Public Understanding of Science 18(1): 23–42.
Strauss, Valerie. 2011. “The letter from assessment experts the N.Y. Regents ignored.” The Washington Post. May 22. https://www.washingtonpost.com/blogs/answer-sheet/post/the-letter-from-assessment-experts-the-ny-regents-ignored/2011/05/21/AFJHIA9G_blog.html?utm_term=.2649802cc7e3. Accessed 2 Aug 2018.
Strauss, Valerie. 2012. “Researchers blast Chicago teacher evaluation reform.” The Washington Post. March 28. https://www.washingtonpost.com/blogs/answer-sheet/post/researchers-blast-chicago-teacher-evaluation-reform/2012/03/28/gIQApdOfgS_blog.html?utm_term=.69ce6632211a. Accessed 2 Aug 2018.
Strauss, Valerie. 2014. “Houston teachers sue over controversial teacher evaluation method.” The Washington Post. April 30. https://www.washingtonpost.com/news/answer-sheet/wp/2014/04/30/houston-teachers-sue-over-controversial-teacher-evaluation-method/?utm_term=.968840b07a48. Accessed 2 Aug 2018.
Weinberg, Alvin M. 1974. Science and trans-science. Minerva 10(2): 209–222.
Wright, Will. 1992. Wild knowledge: Science, language, and social life in a fragile environment. Minneapolis, MN: University of Minnesota Press.
Wynne, Brian. 1992. Uncertainty and environmental learning: Reconceiving science and policy in the preventive paradigm. Global Environmental Change 2: 111–127.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Tobiason, G. Countering Expert Uncertainty: Rhetorical Strategies from the Case of Value-Added Modeling in Teacher Evaluation. Minerva 57, 109–126 (2019). https://doi.org/10.1007/s11024-018-9359-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11024-018-9359-z