Journal of Science Education and Technology

, Volume 23, Issue 1, pp 82–97 | Cite as

The Use and Effectiveness of an Argumentation and Evaluation Intervention in Science Classes

  • Janis A. Bulgren
  • James D. Ellis
  • Janet G. Marquis


This study explored teachers’ use of the Argumentation and Evaluation Intervention (AEI) and associated graphic organizer to enhance the performance of students in middle and secondary science classrooms. The results reported here are from the third year of a design study during which the procedures were developed in collaboration with teachers. A quasi-experimental pretest–posttest design with 8 experimental and 8 control teachers was used with a total of 282 students. An open-ended test assessed students’ abilities to evaluate a scientific argument made in an article. The students were asked to identify the claim and its qualifiers, identify and evaluate the evidence given for the claim, examine the reasoning in support of the claim, consider counterarguments, and construct and explain a conclusion about the claim. The quality of students’ responses was assessed using a scoring rubric for each step of the argumentation process. Findings indicated a significantly higher overall score and large effect size in favor of students who were instructed using the AEI compared to students who received traditional lecture–discussion instruction. Subgroup and subscale scores are also presented. Teacher satisfaction and student satisfaction and confidence levels are reported.


Argumentation Higher-order thinking Middle- and secondary-level science instruction Graphic organizer Design study Student diversity 



This material is based upon work supported by the National Science Foundation under Grant Number 0554414. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.


  1. Achieve (2013) The next generation science standards. Retrieved from
  2. Bannan-Ritland B (2003) The role of design in research: the integrative learning design framework. Educ Res 32(1):21–24. doi: 10.3102/0013189X032001021 Google Scholar
  3. Bricker LA, Bell P (2008) Conceptualizations of argumentation from science studies and the learning sciences and their implications for the practices of science education. Sci Educ 92(3):473–498CrossRefGoogle Scholar
  4. Bulgren JA, Ellis JD (2012) Argumentation and evaluation intervention in science classes: teaching and learning with Toulmin. In: Kline MS (ed) Perspectives on scientific argumentation: theory, practice, and research. Springer, New York, NY, pp 135–154Google Scholar
  5. Common Core State Standards Initiative (CSSS) (2010) Common core state standards for English language arts & literacy in history/social studies, science, and technical subjects. Retrieved from
  6. Conley DT (2008) College knowledge: what it really takes for students to succeed and what we can do to get them ready. Wiley, New York, NYGoogle Scholar
  7. Driver R, Newton P, Osborne D (2000) Establishing the norms of scientific argumentation in classrooms. Sci Educ 84(3):287–312CrossRefGoogle Scholar
  8. Duschl RA, Osborne J (2002) Supporting and promoting argumentation discourse in science education. Stud Sci Educ 38:39–72CrossRefGoogle Scholar
  9. Ellis JD, Bulgren JA (2009) Year 3 report submitted to the National Science Foundation. University of Kansas Center for Research on Learning, LawrenceGoogle Scholar
  10. Erduran S, Simon S, Osborne J (2004) TAPping into argumentation: developments in the application of Toulmin’s argument pattern for studying science discourse. Sci Educ 88:915–933. doi: 10.1002/sce.20012 CrossRefGoogle Scholar
  11. Heller R, Greenleaf C (2007) Literacy instruction in the content areas: getting to the core of middle and high school improvement. Alliance for Excellent Education, Washington, DCGoogle Scholar
  12. Jacobs VA (2008) Adolescent literacy: putting the crisis in context. Harv Educ Rev 78(1):7–39Google Scholar
  13. Jimenez-Alexandre MP, Erduran S (2007) Argumentation in science education: an overview. Contemp Trends Issues Sci Educ 35(1):915–933. doi: 10.1007/978-1-4020-6670-2_1 Google Scholar
  14. Kamil ML, Borman GD, Dole J, Kral CC, Salinger T, Torgesen J (2008) Improving adolescent literacy: effective classroom and intervention practices: a practice guide (NCEE #2008-4027). U.S. Department of Education, National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, Washington, DCGoogle Scholar
  15. Kelly GJ, Regev J, Prothero W (2007) Analysis of lines of reasoning in written argumentation. In: Argumentation in science education. Springer, Netherlands, pp 137–158Google Scholar
  16. Kelly AE, Baek JY, Lesh RA, Bannan-Ritland B (2008) Enabling innovations in education and systemizing their impact. In: Kelly AE, Lesh RA, Baek JY (eds) Handbook of design research methods in education. Routledge, New York, pp 3–18Google Scholar
  17. Klahr D, Fay AL, Dunbar K (1993) Heuristics for scientific experimentation: a developmental study. Cogn Psychol 25(1):111–146CrossRefGoogle Scholar
  18. Kolsto SD, Ratcliffe M (2007) Social aspects of argumentation. Argum Sci Educ 35(2):117–136. doi: 10.1007/978-1-4020-6670-2_6 CrossRefGoogle Scholar
  19. Koslowski B, Okagaki L, Lorenz C, Umbach D (1989) When covariation is not enough: the role of causal mechanism, sampling method, and sample size in causal reasoning. Child Dev 60(6):1316–1327CrossRefGoogle Scholar
  20. Krajcik J, McNeill KL, Reiser BJ (2007) Learning-goals-driven design model: developing curriculum materials that align with national standards and incorporate project-based pedagogy, Wiley InterScience. Retrieved from doi: 10.1002/sce.20240
  21. Kuhn D (1991) The skills of argument. Cambridge University Press, Cambridge, MACrossRefGoogle Scholar
  22. Kuhn D, Amsel E, O’Loughlin M (1988) The development of scientific thinking skills. Academic Press, New York, NYGoogle Scholar
  23. Kuhn D, Garcia-Mila M, Zohar A, Andersen C (1995) Strategies of knowledge acquisition. Monogr Soc Res Child Dev 60(4):1–127CrossRefGoogle Scholar
  24. Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33(1):159–174CrossRefGoogle Scholar
  25. Lawson AE (2003) The nature and development of hypothetico-predictive argumentation with implications for science teaching. International Journal of Science Education 25(11):1387–1408CrossRefGoogle Scholar
  26. Lenz BK, Bulgren J (2013) Improving academic outcomes in content areas. In: Cook BG, Tankersley M (eds) Research-based practices in special education. Pearson, Upper Saddle River, NJ, pp 98–115 (Reprinted from Research-based for improving outcomes in academics, pp. 95–112, by D. J. Chard, B. G. Cook, & M. Tankersley, Eds., 2013, Upper Saddle River, NJ: Pearson)Google Scholar
  27. Linn MC, Clark D, Slotta JD (2003) WISE design for knowledge integration. Sci Educ 87:517–538CrossRefGoogle Scholar
  28. National Academy Press (2012) A framework for K-12 science education: practices, crosscutting concepts, and core ideas. Washington, DCGoogle Scholar
  29. National Research Council (2012) Education for life and work: developing transferable knowledge and skills in the 21st century. The National Academies Press, Washington, DCGoogle Scholar
  30. Norton-Meier L, Hand B, Hockenberry L, Wise K (2008) Questions, claims, and evidence: the important place of argument in children’s science writing. NSTA Press, Arlington, VAGoogle Scholar
  31. Pashler H, Bain PM, Bottge BA, Graesser A, Koedinger K, McDaniel M, Metcalfe J (2007) Organizing instruction and study to improve student learning. IES Practice Guide (NCER 2007–2004). National Center for Education Research, Jessup, MDGoogle Scholar
  32. Penner D, Klahr D (1996) The interaction of domain-specific knowledge and domain-general discovery strategies: a study with sinking objects. Child Dev 67:2709–2727CrossRefGoogle Scholar
  33. Reiser B, Tabak I, Sandoval WA, Smith BK, Steinmuller F, Leone AJ (2001) BGuILE: strategic and conceptual scaffolds for scientific inquiry in biology classrooms. In: Carver SM, Klahr D (eds) Cognition and instruction twenty-five years of progress. Erlbaum, Mahwah, NJ, pp 263–306Google Scholar
  34. Reznitskaya A, Anderson RC (2002) The argument schema and learning to reason. In: Block CC, Pressley M (eds) Comprehension instruction: research-based best practices. Guilford Press, New York, NY, pp 319–334Google Scholar
  35. Sadler TD (2004) Informal reasoning regarding socioscientific issues: a critical review of research. J Res Sci Teach 4:513–536. doi: 10.1002/tea.2009 CrossRefGoogle Scholar
  36. Sandoval WA, Millwood KA (2007) What can argumentation tell us about epistemology? In: Argumentation in science education. Springer, Netherlands, pp 71–88Google Scholar
  37. Schauble L (1996) The development of scientific reasoning in knowledge-rich contexts. Dev Psychol 32:102–119CrossRefGoogle Scholar
  38. Toulmin S, Rieke R, Janik A (1984) An introduction to reasoning. MacMillan, New York, NYGoogle Scholar
  39. U.S. Department of Education, Office of Planning, Evaluation and Policy Development (2010) ESEA blueprint for reform. Author, Washington, DCGoogle Scholar
  40. Vygotsky L (1978) Mind in society. Harvard University Press, Boston, MAGoogle Scholar
  41. What Works Clearinghouse (2010) Procedures and standards handbook (version 2.1). Retrieved from
  42. Ysseldyke J (2009) When politics trumps science: generalizations from a career of research on assessment, decision making, and public policy. Communique 38(4):6–8Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Janis A. Bulgren
    • 1
  • James D. Ellis
    • 2
  • Janet G. Marquis
    • 3
  1. 1.Center for Research on LearningUniversity of KansasLawrenceUSA
  2. 2.School of EducationUniversity of KansasLawrenceUSA
  3. 3.Schiefelbusch Life Span InstituteUniversity of KansasLawrenceUSA

Personalised recommendations