Skip to main content
SpringerLink
Log in

Assessing Science Inquiry and Reasoning Using Dynamic Visualizations and Interactive Simulations

  • Chapter
  • First Online:
Learning from Dynamic Visualization

Abstract

How can we leverage dynamic visualizations and interactive simulations to assess complex science learning? Science systems involve many dynamic processes, and visualizations of these processes are central components of scientific thinking and communication. Although multiple-choice items may effectively measure declarative knowledge such as scientific facts or definitions, they fail to capture evidence of science inquiry practices such as making observations, collecting data, or modeling dynamic systems. As computers become more widely accessible, interactive, simulation-based assessments have the promise of capturing information about students’ ability to apply these more complex science practice skills. Tasks using dynamic visualizations and simulations are likely to provide more valid measures of the scientific proficiencies called for internationally by science educators, such as those reflected in the United States’ National Assessment of Educational Progress and the Next Generation Science Standards. In this chapter, we describe the range of science content knowledge and practice skills we seek to measure, outline research-based assessment design principles, and review a test creation process that mitigates the potential limitations of dynamic assessments. Next, we provide examples of how the stimulus and response affordances of dynamic visualizations allow us to assess scientific inquiry and reasoning skills. Finally, we provide findings from an efficacy study that support the claim that dynamic and interactive assessments provide better measures of student proficiency with complex scientific inquiry and reasoning. Throughout the chapter, we give concrete examples of the principles and processes in the context of two science assessment platforms, SimScientists and ChemVLab+.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    From Quellmalz et al., 2013. Copyright 2014 by American Psychological Association. Adapted with permission.

References

  • Adams, W. K., Reid, S., Lemaster, R., McKagan, S. B., Perkins, K. K., Dubson, M., et al. (2008). A study of educational simulations part 1: Engagement and learning. Journal of Interactive Learning Research, 19, 397–419.

    Google Scholar 

  • Ainsworth, S. (2008). The educational value of multiple-representations when learning complex scientific concepts. In J. K. Gilbert, M. Reiner, & M. Nakhleh (Eds.), Visualization: Theory and practice in science education (pp. 191–208). New York: Springer.

    Chapter  Google Scholar 

  • BĂ©trancourt, M. (2005). The animation and interactivity principles in multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 287–296). New York: Cambridge University Press.

    Chapter  Google Scholar 

  • Boucheix, J.-M. (2008). Young learners’ control of technical animations. In R. K. Lowe & W. Schnotz (Eds.), Learning with animation: Research implications for design (pp. 208–234). New York: Cambridge University Press.

    Google Scholar 

  • Boucheix, J.-M., & Lowe, R. K. (2010). An eye tracking comparison of external pointing cues and internal continuous cues in learning with complex animations. Learning and Instruction, 20, 123–135.

    Article  Google Scholar 

  • Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press.

    Google Scholar 

  • Brennan, R. (2001). mGENOVA. Retrieved from http://www.education.uiowa.edu/centers/casma/computer-programs.aspx.

  • Buckley, B. C., Gobert, J., Horwitz, P., & O’Dwyer, L. (2010). Looking inside the black box: Assessing model-based learning and inquiry in BioLogica. International Journal of Learning Technologies, 5, 166–190.

    Article  Google Scholar 

  • Buckley, B. C., & Quellmalz, E. S. (2013). Supporting and assessing complex biology learning with computer-based simulations and representations. In D. Treagust & C. Y. Tsui (Eds.), Multiple representations in biological education (pp. 247–267). New York: Springer.

    Chapter  Google Scholar 

  • Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81–105.

    Article  Google Scholar 

  • Clark, R. C., & Mayer, R. E. (2011). E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning. San Francisco, CA: Pfeiffer.

    Book  Google Scholar 

  • Cuadros, J., Leinhardt, G., & Yaron, D. (2007). One firm spot: The role of homework as lever in acquiring conceptual and performance competence in college chemistry. Journal of Chemical Education, 84, 1047–1052.

    Article  Google Scholar 

  • Davenport, J. L., Rafferty, A., Timms, M. J., Yaron, D., & Karabinos, M. (2012a). ChemVLab+: Evaluating a virtual lab tutor for high school chemistry. In J. van Aalst, K. Thompson, M. J. Jacobson, & P. Reimann (Eds.), Proceedings of the 10th international conference of the learning sciences (pp. 381–385). Sydney, Australia: International Society of the Learning Sciences.

    Google Scholar 

  • Davenport, J. L., Timms, M. J., Yaron, D., & Karabinos, M. (2012b) ChemVlab+: Supporting science learning using a virtual chemistry lab with embedded assessments. Paper presented at the 2012 Annual Meeting of the American Educational Research Association in Vancouver, BC.

    Google Scholar 

  • Davenport, J.L., Powers, J., Rafferty, A., Timms, M., Karabinos, M., & Yaron, D. (2014a, April). Classroom factors and students learning with online virtual chemistry lab activities. Paper presented at the 2014 Annual Meeting of the American Educational Research Association in Philadelphia, PA.

    Google Scholar 

  • Davenport, J. L., Rafferty, A., Yaron, D, Karabinos, M., & Timms, M. (2014b, April). ChemVLab+: Simulation-based lab activities to support chemistry learning. Paper presented at the 2014 Annual Meeting of the American Educational Research Association in Philadelphia, PA.

    Google Scholar 

  • De Jong, T. (2006). Technological advances in inquiry learning. Science, 312(5773), 532–533.

    Article  Google Scholar 

  • De Koning, B. B., & Jarodzka, H. (2017). Attention guidance strategies for supporting learning from dynamic visualizations. In R. Lowe & R. Ploetzner (Eds.), Learning from dynamic visualization – Innovations in research and application. Berlin: Springer (this volume).

    Google Scholar 

  • DeBoer, G. E., Quellmalz, E. S., Davenport, J. L., Timms, M. J., Herrmann-Abell, C. F., Buckley, B. C., et al. (2014). Comparing three online testing modalities: Using static, active, and interactive online testing modalities to assess middle school students’ understanding of fundamental ideas and use of inquiry skills related to ecosystems. Journal of Research in Science Teaching, 51, 523–554.

    Article  Google Scholar 

  • Gallegos, L., Jerezano, M., & Flores, F. (1994). Preconceptions and relations used by children in the construction of food chains. Journal of Research in Science Teaching, 31, 259–272.

    Article  Google Scholar 

  • Goldman, S. R. (2003). Learning in complex domains: When and why do multiple representations help? Learning and Instruction, 13, 239–244.

    Article  Google Scholar 

  • Hegarty, M. (2004). Dynamic visualizations and learning: Getting to the difficult questions. Learning and Instruction, 14, 343–351.

    Article  Google Scholar 

  • Hegarty, M., & Just, M. A. (1993). Constructing mental models of machines from text and diagrams. Journal of Memory and Language, 32, 717–742.

    Article  Google Scholar 

  • Hmelo-Silver, C. E., Jordan, R., Liu, L., Gray, S., Demeter, M., Rugaber, S., et al. (2008). Focusing on function: Thinking below the surface of complex science systems. Science Scope, 31(9), 27–35.

    Google Scholar 

  • Horwitz, P., Gobert, J. D., Buckley, B. C., & O’Dwyer, L. M. (2010). Learning genetics with dragons: From computer-based manipulatives to hypermodels. In M. J. Jacobson & P. Reimann (Eds.), Designs for learning environments of the future: International perspectives from the learning sciences (pp. 61–87). New York: Springer.

    Chapter  Google Scholar 

  • Ioannidou, A., Repenning, A., Webb, D., Keyser, D., Luhn, L., & Daetwyler, C. (2010). Mr. Vetro: A collective simulation for teaching health science. International Journal of Computer-Supported Collaborative Learning, 5, 141–166.

    Article  Google Scholar 

  • Krajcik, J., Marx, R., Blumenfeld, P., Soloway, E., & Fishman, B. (2000). Inquiry-based science supported by technology: Achievement and motivation among urban middle school students. Paper presented at the American Educational Research Association, New Orleans, LA.

    Google Scholar 

  • Kriz, S., & Hegarty, M. (2007). Top-down and bottom-up influences on learning from animations. International Journal of Human-Computer Studies, 65, 911–930.

    Article  Google Scholar 

  • KĂĽhl, T., Scheiter, K., Gerjets, P., & Edelmann, J. (2011). The influence of text modality on learning with static and dynamic visualizations. Computers in Human Behavior, 27, 29–35.

    Article  Google Scholar 

  • Larkin, J., & Simon, H. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 11, 65–100.

    Article  Google Scholar 

  • Lee, H., Plass, J. L., & Homer, B. D. (2006). Optimizing cognitive load for learning from computer-based science simulations. Journal of Educational Psychology, 98, 902–913.

    Article  Google Scholar 

  • Lehrer, R., Schauble, L., Strom, D., & Pligge, M. (2001). Similarity of form and substance: Modeling material kind. In D. K. S. Carver (Ed.), Cognition and instruction: 25 years of progress (pp. 39–74). Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Loehlin, J. C. (1998). Latent variable models: An introduction to factor, path, and structural analysis. Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Lowe, R. K. (2008). Learning from animation: Where to look, when to look. In R. K. Lowe & W. Schnotz (Eds.), Learning with animation: Research implications for design (pp. 49–70). New York: Cambridge University Press.

    Google Scholar 

  • Lowe, R., & Boucheix, J.-M. (2017). A composition approach to design of educational animations. In R. Lowe & R. Ploetzner (Eds.), Learning from dynamic visualization – Innovations in research and application. Berlin: Springer (this volume).

    Chapter  Google Scholar 

  • Lowe, R., Boucheix, J.-M., & Fillisch, B. (2017). Demonstration tasks for assessment. In R. Lowe & R. Ploetzner (Eds.), Learning from dynamic visualization – Innovations in research and application. Berlin: Springer (this volume).

    Chapter  Google Scholar 

  • Lowe, R., & Schnotz, W. (Eds.). (2008). Learning with animation: Research implications for design. New York: Cambridge University Press.

    Google Scholar 

  • Lowe, R. K., & Schnotz, W. (2014). Animation principles in multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 279–315). New York: Cambridge University Press.

    Google Scholar 

  • Mayer, R. E. (Ed.). (2005). The Cambridge handbook of multimedia learning. New York: Cambridge University Press.

    Google Scholar 

  • Mayer, R. E. (Ed.). (2014). The Cambridge handbook of multimedia learning (2nd ed.). New York: Cambridge University Press.

    Google Scholar 

  • Mayer, R. E., & Fiorella, L. (2014). Principals for reducing extraneous processing in multimedia learning: Coherence, signaling, redundancy, spatial contiguity, and temporal contiguity principles. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 279–315). New York: Cambridge University Press.

    Chapter  Google Scholar 

  • Mayer, R. E., & Johnson, C. (2008). Revising the redundancy principle in multimedia learning. Journal of Educational Psychology, 100, 380–386.

    Article  Google Scholar 

  • Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2003). A brief introduction to evidence-centered design (Research Report No. RR-03-16). Princeton, NJ: Educational Testing Service.

    Google Scholar 

  • Mullis, I. V. S., & Martin, M. O. (Eds.). (2013). TIMSS 2015 assessment frameworks. Chestnut Hill, MA: TIMSS & PIRLS International Study Center at Boston College.

    Google Scholar 

  • National Assessment Governing Board (NAGB). (2010). Science framework for the 2011 national assessment of educational progress. Washington, DC: National Assessment Governing Board.

    Google Scholar 

  • National Center for Education Statistics (NCES). (2012). The nation’s report card: science in action: Hands-on and interactive computer tasks from the 2009 science assessment (NCES 2012–468). Washington, DC: Institute of Education Sciences, U.S. Department of Education.

    Google Scholar 

  • NGSS Lead States (NGSS). (2013). Next generation science standards: For states, by states. Washington, DC: The National Academies Press.

    Google Scholar 

  • Organization for Economic Cooperation and Development (OECD). (2009). PISA 2009 assessment framework: Key competencies in reading, mathematics and science. Paris, France: OECD.

    Google Scholar 

  • Papaevripidou, M., Constantinou, C. P., & Zacharia, Z. C. (2007). Modeling complex marine ecosystems: An investigation of two teaching approaches with fifth graders. Journal of Computer Assisted Learning, 23, 145–157.

    Article  Google Scholar 

  • Pedone, R., Hummel, J. E., & Holyoak, K. J. (2001). The use of diagrams in analogical problem solving. Memory and Cognition, 29, 214–221.

    Article  Google Scholar 

  • Pellegrino, J. W. (2013). Proficiency in science: Assessment challenges and opportunities. Science, 340(6130), 320–323.

    Article  Google Scholar 

  • Petre, M., & Green, T. R. G. (1993). Learning to read graphics: Some evidence that “seeing” an information display is an acquired skill. Journal of Visual Languages & Computing, 4, 55–70.

    Article  Google Scholar 

  • Plass, J. L., Homer, B. D., & Hayward, E. O. (2009). Design factors for educationally effective animations and simulations. Journal of Computing in Higher Education, 21(1), 31–61.

    Article  Google Scholar 

  • Quellmalz, E. S., Davenport, J. L., Timms, M. J., DeBoer, G. E., Jordan, K. A., Huang, C., et al. (2013). Next-generation environments for assessing and promoting complex science learning. Journal of Educational Psychology, 5, 1100.

    Article  Google Scholar 

  • Quellmalz, E. S., & Pellegrino, J. W. (2009). Technology and testing. Science, 323(5910), 75–79.

    Article  Google Scholar 

  • Quellmalz, E. S., Timms, M. J., & Buckley, B. (2010). The promise of simulation-based science assessment: The Calipers project. International Journal of Learning Technology, 5, 243–263.

    Article  Google Scholar 

  • Quellmalz, E. S., Timms, M. J., Silberglitt, M. D., & Buckley, B. C. (2011). Science assessments for all: Integrating science simulations into balanced state science assessment systems. Journal of Research in Science Teaching, 49, 363–393.

    Article  Google Scholar 

  • Reiner, M., & Eilam, B. (2001). Conceptual classroom environment: A system view of learning. International Journal of Science Education, 23, 551–568.

    Article  Google Scholar 

  • Rieber, L. P., Tzeng, S., & Tribble, K. (2004). Discovery learning, representation, and explanation within a computer-based simulation. Computers and Education, 27, 45–58.

    Article  Google Scholar 

  • Schwartz, D. L., & Black, T. (1999). Inferences through imagined actions: Knowing by simulated doing. Journal of Experimental Psychology: Learning, Memory & Cognition, 25, 116–136.

    Google Scholar 

  • Schwartz, D. L., & Heiser, J. (2006). Spatial representations and imagery in learning. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 283–298). New York: Cambridge University Press.

    Google Scholar 

  • Schweingruber, H., Keller, T., & Quinn, H. (Eds.). (2012). A framework for K-12 Science Education: Practices, crosscutting concepts, and core ideas. Washington, DC: National Academies Press.

    Google Scholar 

  • Slotta, J. D., & Chi, M. T. H. (2006). Helping students understand challenging topics in science through ontology training. Cognition and Instruction, 24, 261–289.

    Article  Google Scholar 

  • Stewart, J., Cartier, J. L., & Passmore, C. M. (2005). Developing understanding through model-based inquiry. In M. S. Donovan & J. D. Bransford (Eds.), How students learn (pp. 515–565). Washington, DC: The National Academies Press.

    Google Scholar 

  • Tversky, B., Heiser, J., Lozano, S., MacKenzie, R., & Morrison, J. (2008). Enriching animations. In R. K. Lowe & W. Schnotz (Eds.), Learning with animation: Research implications for design (pp. 263–285). New York: Cambridge University Press.

    Google Scholar 

  • Van MerriĂ«nboer, J. J. G., & Kester, L. (2005). The four-component instructional design model: Multimedia principles in environments for complex learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 71–93). New York: Cambridge University Press.

    Chapter  Google Scholar 

  • Vattam, S. S., Goel, A. K., Rugaber, S., Hmelo-Silver, C. E., Jordan, R., Gray, S., et al. (2011). Understanding complex natural systems by articulating structure-behavior- function models. Journal of Educational Technology & Society, 14, 66–81.

    Google Scholar 

  • Webb, N. M., & Shavelson, R. J. (2005). Generalizability theory: Overview. In B. Everitt & D. Howell (Eds.), Encyclopedia of statistics in behavioral science (Vol. 2, pp. 717–719). Chichester, UK: Wiley.

    Google Scholar 

  • Zacharia, Z. C. (2007). Comparing and combining real and virtual experimentation: An effort to enhance students’ conceptual understanding of electric circuits. Journal of Computer Assisted Learning, 23, 120–132.

    Article  Google Scholar 

Download references

Acknowledgements

This material chapter is based upon work supported by the National Science Foundation under Grant No. DRL-0814776 awarded to WestEd, Edys Quellmalz, Principal Investigator and work supported by the Institute of Education Sciences, through Grant R305A100069 to WestEd, Jodi Davenport, Principal Investigator. Any opinions, findings and conclusions or recommendations expressed in this material chapter are those of the author(s) and do not necessarily reflect the views of the National Science Foundation or the U.S. Department of Education.

Author information

Authors and Affiliations

  1. WestEd, Oakland, CA, USA

    Jodi L. Davenport

  2. WestEd, Redwood City, CA, USA

    Edys S. Quellmalz

Authors
  1. Jodi L. Davenport
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Edys S. Quellmalz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jodi L. Davenport .

Editor information

Editors and Affiliations

  1. School of Education, Curtin University School of Education, Bentley, West Australia, Australia

    Richard Lowe

  2. University of Education, Freiburg, Germany

    Rolf Ploetzner

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Davenport, J.L., Quellmalz, E.S. (2017). Assessing Science Inquiry and Reasoning Using Dynamic Visualizations and Interactive Simulations. In: Lowe, R., Ploetzner, R. (eds) Learning from Dynamic Visualization. Springer, Cham. https://doi.org/10.1007/978-3-319-56204-9_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-56204-9_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-56202-5

  • Online ISBN: 978-3-319-56204-9

  • eBook Packages: EducationEducation (R0)

Publish with us

Policies and ethics

Search

Navigation