Journal of Science Teacher Education

, Volume 27, Issue 2, pp 137–164 | Cite as

Measuring Science Instructional Practice: A Survey Tool for the Age of NGSS

  • Kathryn N. Hayes
  • Christine S. Lee
  • Rachelle DiStefano
  • Dawn O’Connor
  • Jeffery C. Seitz


Ambitious efforts are taking place to implement a new vision for science education in the United States, in both Next Generation Science Standards (NGSS)-adopted states and those states creating their own, often related, standards. In-service and pre-service teacher educators are involved in supporting teacher shifts in practice toward the new standards. With these efforts, it will be important to document shifts in science instruction toward the goals of NGSS and broader science education reform. Survey instruments are often used to capture instructional practices; however, existing surveys primarily measure inquiry based on previous definitions and standards and with a few exceptions, disregard key instructional practices considered outside the scope of inquiry. A comprehensive survey and a clearly defined set of items do not exist. Moreover, items specific to the NGSS Science and Engineering practices have not yet been tested. To address this need, we developed and validated a Science Instructional Practices survey instrument that is appropriate for NGSS and other related science standards. Survey construction was based on a literature review establishing key areas of science instruction, followed by a systematic process for identifying and creating items. Instrument validity and reliability were then tested through a procedure that included cognitive interviews, expert review, exploratory and confirmatory factor analysis (using independent samples), and analysis of criterion validity. Based on these analyses, final subscales include: Instigating an Investigation, Data Collection and Analysis, Critique, Explanation and Argumentation, Modeling, Traditional Instruction, Prior Knowledge, Science Communication, and Discourse.


Inquiry Instructional practice Survey tool Next Generation Science Standards 



This work was supported by the National Science Foundation Grant No. 0962804.


  1. Abd-El-Khalick, F., Boujaoude, S., Duschl, R. A., Lederman, N. G., Mamlok-Naaman, R., Hofstein, A., & Tuan, H.-L. (2004). Inquiry in science education: International perspectives. Science Education, 88, 397–419.CrossRefGoogle Scholar
  2. Allen, M. J., & Yen, W. M. (1979). Introduction to measurement theory. Monterey, CA: Brooks-Cole.Google Scholar
  3. Anderson, R. D. (2002). Reforming science teaching: What research says about inquiry. Journal of Science Teacher Education, 13, 1–12.CrossRefGoogle Scholar
  4. Au, W. (2007). High-stakes testing and curricular control: A qualitative metasynthesis. Educational Researcher, 36(5), 258–267.CrossRefGoogle Scholar
  5. Bandalos, D. L., & Finney, S. J. (2010). Factor analysis: Exploratory and confirmatory. In G. R. Hancock & R. O. Mueller (Eds.), The reviewer’s guide to quantitative methods in the social sciences (pp. 93–114). New York: Routledge.Google Scholar
  6. Banilower, E. R., Heck, D. J., & Weiss, I. R. (2007). Can professional development make the vision of the standards a reality? The impact of the national science foundation’s local systemic change through teacher enhancement initiative. Journal of Research in Science Teaching, 44, 375–395.CrossRefGoogle Scholar
  7. Banilower, E. R., Smith, S. P., Weiss, I. R., Malzahn, K. A., Campbell, K. M., & Weis, A. M. (2013). Report of the 2012 national survey of science and mathematics education. Chapell Hill, NC: Horizon Research.Google Scholar
  8. Bartholomew, H., Osborne, J., & Ratcliffe, M. (2004). Teaching students “ideas-about-science”: Five dimensions of effective practice. Science Education, 88, 655–682.CrossRefGoogle Scholar
  9. Burstein, L., McDonnell, L. M., Van Winkle, J., Ormseth, T., Mirocha, J., & Guitón, G. (1995). Validating national curriculum indicators. Santa Monica, CA: RAND Corporation.Google Scholar
  10. Calabrese Barton, A. (2002). Learning about transformative research through others’ stories: What does it mean to involve “others” in science education reform? Journal of Research in Science Teaching, 39, 110–113.CrossRefGoogle Scholar
  11. Calabrese Barton, A., Tan, E., & Rivet, A. (2008). Creating hybrid spaces for engaging school science among urban middle school girls. American Educational Research Journal, 45(1), 68–103.CrossRefGoogle Scholar
  12. Campbell, T., Abd-Hamid, N. H., & Chapman, H. (2010). Development of instruments to assess teacher and student perceptions of inquiry experiences in science classrooms. Journal of Science Teacher Education, 21, 13–30.CrossRefGoogle Scholar
  13. Canale, M., & Swain, M. (1980). Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1, 1–47.CrossRefGoogle Scholar
  14. Capps, D. K., & Crawford, B. A. (2013). Inquiry-based instruction and teaching about nature of science: Are they happening? Journal of Science Teacher Education, 24, 497–526.CrossRefGoogle Scholar
  15. Center on Education Policy. (2007). Choices, changes, and challenges: Curriculum and instruction in the NCLB era. Washington, DC: Center on Education Policy.Google Scholar
  16. Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Orlando, FL: Holt, Rinehart and Winston.Google Scholar
  17. Cuban, L. (2013). Inside the black box of classroom practice: Change without reform in American education. Cambridge, MA: Harvard Education Press.Google Scholar
  18. Desimone, L. M., & Le Floch, K. C. (2004). Are we asking the right questions? Using cognitive interviews to improve surveys in education research. Educational Evaluation and Policy Analysis, 26, 1–22.CrossRefGoogle Scholar
  19. Desimone, L. M., Porter, A. C., Garet, M. S., Yoon, K. S., & Birman, B. F. (2002). Effects of professional development on teachers’ instruction: Results from a three-year longitudinal study. Educational Evaluation and Policy Analysis, 24, 81–112.CrossRefGoogle Scholar
  20. Dorph, R., Sheilds, P., Tiffany-Morales, J., Hartry, A., & McCaffrey, T. (2011). High hopes-few opportunities: The status of elementary science education in California. Sacramento, CA: The Center for the Future of Teaching and Learning at WestEd.Google Scholar
  21. Driver, R., Asoko, H., Leach, J., Scott, P., & Mortimer, E. (1994). Constructing scientific knowledge in the classroom. Educational Researcher, 23(7), 5–12.CrossRefGoogle Scholar
  22. Driver, R., Newton, P., & Osborne, J. (2000). Establishing the norms of scientific argumentation in classrooms. Science Education, 84, 287–312.CrossRefGoogle Scholar
  23. Duschl, R. A., & Osborne, J. (2002). Supporting and promoting argumentation discourse in science education. Studies in Science Education, 38, 39–72.CrossRefGoogle Scholar
  24. Forbes, C. T., Biggers, M., & Zangori, L. (2013). Investigating essential characteristics of scientific practices in elementary science learning environments: The practices of science observation protocol (P-SOP). School Science and Mathematics, 113, 180.CrossRefGoogle Scholar
  25. Garet, M. S., Birman, B. F., Porter, A. C., Desimone, L., & Herman, R. (1999). Designing effective professional development: Lessons from the Eisenhower Program [and] technical appendices. Jessup, MD: Editorial Publications Center, US Department of Education.Google Scholar
  26. Germuth, A., Banilower, E., & Shimkus, E. (2003). Test-retest reliability of the Local Systemic Change teacher questionnaire. Chapel Hill, NC: Horizon Research.Google Scholar
  27. Gogol, K., Brunner, M., Goetz, T., Martin, R., Ugen, S., Keller, U., … Preckel, F. (2014). ”My Questionnaire is Too Long!” The assessments of motivational-affective constructs with three-item and single-item measures. Contemporary Educational Psychology, 34, 188–205.CrossRefGoogle Scholar
  28. Hayes, K. N., & Trexler, C. J. (2016). Testing predictors of instructional practice in elementary science education: The significant role of accountability. Science Education. (in press).Google Scholar
  29. Hill, L., & Betz, D. (2005). Revisiting the retrospective pretest. American Journal of Evaluation, 26(4), 501–517.CrossRefGoogle Scholar
  30. Hogan, K., Nastasi, B. K., & Pressley, M. (1999). Discourse patterns and collaborative scientific reasoning in peer and teacher-guided discussions. Cognition and Instruction, 17, 379.CrossRefGoogle Scholar
  31. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55.CrossRefGoogle Scholar
  32. Huffman, D., Thomas, K., & Lawrenz, F. (2003). Relationship between professional development, teachers’ instructional practices, and the achievement of students in science and mathematics. School Science and Mathematics, 103, 378–387.CrossRefGoogle Scholar
  33. Jiménez-Aleixandre, M. P., & Erduran, S. (2007). Argumentation in science education: An overview. In S. Erduran & M. P. Jiménez-Aleixandre (Eds.), Argumentation in science education: Perspectives from classroom-based research (pp. 3–27). Berlin: Springer.CrossRefGoogle Scholar
  34. Klein, S., Hamilton, L., McCaffrey, D., Stecher, B., Robyn, A., & Burroughs, D. (2000). Teaching practices and student achievement: Report of first-year findings from the “Mosaic” study of Systemic Initiatives in Mathematics and Science. Santa Monica, CA: Rand Corporation.Google Scholar
  35. Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into practice, 41(4), 212–218.CrossRefGoogle Scholar
  36. Kuhn, D. (2015). Thinking together and alone. Educational Researcher, 44, 46–53.CrossRefGoogle Scholar
  37. Lee, C. D., Luykx, A., Buxton, C., & Shaver, A. (2007). The challenge of altering elementary school teachers’ beliefs and practices regarding linguistic and cultural diversity in science instruction. Journal of Research in Science Teaching, 44, 1269–1291.CrossRefGoogle Scholar
  38. Lee, O., Maerten-Rivera, J., Buxton, C., Penfield, R., & Secada, W. G. (2009). Urban elementary teachers’ perspectives on teaching science to English language learners. Journal of Science Teacher Education, 20, 263–286.CrossRefGoogle Scholar
  39. Lemke, J. L. (2001). Articulating communities: Sociocultural perspectives on science education. Journal of Research in Science Teaching, 38(3), 296–316.CrossRefGoogle Scholar
  40. Lemke, J. (2004). The literacies of science. In E. W. Saul (Ed.), Crossing borders in literacy and science instruction: Perspectives on theory and practice. Newark, DE: International Reading Association.Google Scholar
  41. Llewellyn, D. (2013). Teaching high school science through inquiry and argumentation. Thousand Oaks, CA: Corwin.Google Scholar
  42. Marsh, H. W. (1986). Global self-esteem: Its relation to specific facets of self-concept and their importance. Journal of Personality and Social Psychology, 51(6), 1224.CrossRefGoogle Scholar
  43. Marsh, H. W., Hau, K.-T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling, 11, 320–341.CrossRefGoogle Scholar
  44. Marshall, J. C., Smart, J., & Horton, R. M. (2009). The design and validation of EQUIP: An instrument to assess inquiry-based instruction. International Journal of Science and Mathematics Education, 8, 299–321.CrossRefGoogle Scholar
  45. Matsunaga, M. (2010). How to factor-analyze your data right: Do’s, don’ts, and how-to’s. International Journal of Psychological Research, 3(1), 97–110.Google Scholar
  46. McGinn, M. K., & Roth, W. M. (1999). Preparing students for competent scientific practice: Implications of recent research in science and technology studies. Educational Researcher, 28, 14–24.CrossRefGoogle Scholar
  47. McNeill, K. L., & Krajcik, J. (2008). Scientific explanations: Characterizing and evaluating the effects of teachers’ instructional practices on student learning. Journal of Research in Science Teaching, 45, 53–78.CrossRefGoogle Scholar
  48. Moll, L., Amanti, C., Neff, D., & Gonzalez, N. (1992). Funds of knowledge for teaching, using a qualitative approach to connect homes and classrooms. Theory into Practice, 31(2), 132–141.CrossRefGoogle Scholar
  49. National Research Council. (1996). National science education standards. Washington, DC: The National Academies Press.Google Scholar
  50. National Research Council (NRC). (2012). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. Washington, DC: The National Academies Press.Google Scholar
  51. National Research Council (NRC). (2013). Next generation science standards: For states, by states. Washington, DC: The National Academies Press.Google Scholar
  52. Norris, S., Philips, L., & Osborne, J. (2008). Scientific inquiry: The place of interpretation and argumentation. In J. Luft, R. L. Bell, & J. Gess-Newsome (Eds.), Science as inquiry in the secondary setting (pp. 87–98). Arlington, VA: NSTA Press.Google Scholar
  53. President’s Council of Advisors on Science and Technology (PCAST). (2010). Report to the President: Prepare and inspire: K-12 education in science, technology, engineering, and mathematics (STEM) for America’s future. Washington, DC: Executive Office of the President.
  54. Rea, L. M., & Parker, R. A. (2005). Designing and conducting survey research: A comprehensive guide. San Francisco: Jossey-Bass.Google Scholar
  55. Richmond, G., & Striley, J. (1996). Making meaning in classrooms: Social processes in small-group discourse and scientific knowledge building. Journal of Research in Science Teaching, 33(8), 839–858.CrossRefGoogle Scholar
  56. Rosebery, A. S., Warren, B., & Conant, F. R. (1992). Appropriating scientific discourse: Findings from language minority classrooms. The Journal of the Learning Sciences, 2, 61–94.CrossRefGoogle Scholar
  57. Schwarz, C. V., Reiser, B. J., Davis, E. A., Kenyon, L., Achér, A., Fortus, D., … Krajcik, J. (2009). Developing a learning progression for scientific modeling: Making scientific modeling accessible and meaningful for learners. Journal of Research in Science Teaching, 46(6), 632–654.CrossRefGoogle Scholar
  58. Smith, M. S. (2000). Balancing old and new: An experienced middle school teacher’s learning in the context of mathematics instructional reform. The Elementary School Journal, 100(4), 351–375.Google Scholar
  59. Smolleck, L. D., Zembal-Saul, C., & Yoder, E. P. (2006). The development and validation of an instrument to measure preservice teachers’ self-efficacy in regard to the teaching of science as inquiry. Journal of Science Teacher Education, 17, 137–163.CrossRefGoogle Scholar
  60. Stapleton, L. M. (2010). Survey sampling, administration, and analysis. In G. R. Hancock & R. O. Mueller (Eds.), The reviewer’s guide to quantitative methods in the social sciences (pp. 399–412). New York: Routledge.Google Scholar
  61. Stewart, J., Cartier, J. L., & Passmore, C. M. (2005). Developing understanding through model-based inquiry. In S. Donovan & J. D. Bransford (Eds.), How students learn: Science in the classroom (pp. 515–565). Washington, DC: National Academies Press.Google Scholar
  62. Supovitz, J. A., & Turner, H. M. (2000). The effects of professional development on science teaching practices and classroom culture. Journal of Research in Science Teaching, 37(9), 963–980.CrossRefGoogle Scholar
  63. Tekkumru Kisa, M. T., & Stein, M. K. (2015). Learning to see teaching in new ways: A foundation for maintaining cognitive demand. American Educational Research Journal, 52(1), 105–136.CrossRefGoogle Scholar
  64. Tekkumru Kisa, M., Stein, M. K., & Schunn, C. (2015). A framework for analyzing cognitive demand and content-practices integration: Task analysis guide in science. Journal of Research in Science Teaching, 52, 659–685.CrossRefGoogle Scholar
  65. Thorndike, R. M., & Thorndike-Christ, T. (2010). Measurement and evaluation in psychology and education (8th ed.). Boston: Pearson, Merrill.Google Scholar
  66. Walczyk, J. J., & Ramsey, L. L. (2003). Use of learner-centered instruction in college science and mathematics classrooms. Journal of Research in Science Teaching, 40, 566–584.CrossRefGoogle Scholar
  67. Wertsch, J. V. (1985). Vygotsky and the social formation of mind. Cambridge, MA: Harvard University Press.Google Scholar
  68. Zimmerman, C. (2007). The development of scientific thinking skills in elementary and middle school. Developmental Review, 27, 172–223.CrossRefGoogle Scholar

Copyright information

© The Association for Science Teacher Education, USA 2016

Authors and Affiliations

  1. 1.California State UniversityHaywardUSA
  2. 2.Alameda County Office of EducationHaywardUSA

Personalised recommendations