Skip to main content

Advertisement

SpringerLink
Log in

Validation of an Instrument for Measuring Students’ Understanding of Interdisciplinary Science in Grades 4-8 over Multiple Semesters: a Rasch Measurement Study

  • Published:
International Journal of Science and Mathematics Education Aims and scope Submit manuscript

Abstract

So far, not enough effort has been invested in developing reliable, valid, and engaging assessments in school science, especially assessment of interdisciplinary science based on the new Next Generation Science Standards (NGSS). Furthermore, previous tools rely mostly on multiple-choice items and evaluation of student outcome is linked only to their raw scores in standardized tests. In addition, educational research on student science achievement always limits in certain grade/subject according to application of raw scores. This study provides a way to design, validate, and further improve an instrument to assess student understanding in interdisciplinary science across grade and time. Meanwhile, student learning growth is investigated. The data are collected from elementary/middle school student survey from an Interdisciplinary Science and Engineering Partnership (ISEP) in Northeastern part of the USA. The results show a generally good quality of instrument through various aspects of empirical evidence, such as dimensionality, model-data-fit, and validity. Finally, student learning growth in understanding interdisciplinary science shows a sharp increase between elementary and middle school. The study sheds light on developing more reliable and valid instruments in assessing student science understanding based on the new standards and it also provides suggestions of implementation in both educational research and practice.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Baker, T. R. & White, S. H. (2003). The effects of GIS on students’ attitudes, self-efficacy, and achievement in middle school science classrooms. Journal of Geography, 102(6), 243–254.

    Article  Google Scholar 

  • Bond, T. G. & Fox, C. M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.). Mahwah, NJ: Lawrence Erlbaum.

  • Boone, W. J., Staver, J. R. & Yale, M. S. (2014). Rasch analysis in the human sciences. Dordrecht, The Netherlands: Springer.

  • Britner, S. L. & Pajares, F. (2001). Self-efficacy beliefs, motivation, race, and gender in middle school science. Journal of Women and Minorities in Science and Engineering, 7(4), 271–285.

    Google Scholar 

  • Britton, E. D. & Schneider, S. A. (2007). Large-scale assessments in science education. In S. Abell & N. Lederman (Eds.), Handbook of research on science education (pp. 1007–1040). Mahwah, NJ: Lawrence Erlbaum.

  • Chen, Y.-L., Pan, P.-R., Sung, Y.-T. & Chang, K.-E. (2013). Correcting misconceptions on electronics: Effects of a simulation-based learning environment backed by a conceptual change model. Journal of Educational Technology & Society, 16(2), 212–227.

    Google Scholar 

  • Czerniak, C. M. (2007). Interdisciplinary science teaching. In S. Abell & N. Lederman (Eds.), Handbook of research on science education (pp. 537–559). Mahwah, NJ: Lawrence Erlbaum.

  • Davis-Kean, P. E. (2005). The influence of parent education and family income on child achievement: The indirect role of parental expectations and the home environment. Journal of Family Psychology, 19(2), 294–304.

    Article  Google Scholar 

  • Duit, R. & Treagust, D. F. (2003). Conceptual change: A powerful framework for improving science teaching and learning. International Journal of Science Education, 25(6), 671–688.

    Article  Google Scholar 

  • Fox, A. R. (2014). Examination of consistency on the Ohio Achievement Assessments and Ohio Graduation Test. Theses, Dissertations and Capstones, 873. Retrieved from http://mds.marshall.edu/etd/873.

  • Haladyna, T. M. (2012). Developing and validating multiple-choice test items (3rd ed.). Mahwah, NJ: Lawrence Erlbaum.

  • Hill, N. E. & Tyson, D. F. (2009). Parental involvement in middle school: A meta-analytic assessment of the strategies that promote achievement. Developmental Psychology, 45(3), 740–763.

    Article  Google Scholar 

  • Jeynes, W. H. (2007). The relationship between parental involvement and urban secondary school student academic achievement a meta-analysis. Urban Education, 42(1), 82–110.

    Article  Google Scholar 

  • Johnson, P. (1998). Progression in children’s understanding of a ‘basic’ particle theory: A longitudinal study. International Journal of Science Education, 20(4), 393–412.

    Article  Google Scholar 

  • Kahle, J. B., Meece, J. & Scantlebury, K. (2000). Urban African-American middle school science students: Does standards-based teaching make a difference? Journal of Research in Science Teaching, 37(9), 1019–1041.

    Article  Google Scholar 

  • Klassen, S. (2006). Contextual assessment in science education: Background, issues, and policy. Science Education, 90(5), 820–851.

    Article  Google Scholar 

  • Komarraju, M. & Nadler, D. (2013). Self-efficacy and academic achievement: Why do implicit beliefs, goals, and effort regulation matter? Learning and Individual Differences, 25, 67–72.

    Article  Google Scholar 

  • Lawson, A. E. (1993). Deductive reasoning, brain maturation, and science concept acquisition: Are they linked? Journal of Research in Science Teaching, 30(9), 1029–1051.

    Article  Google Scholar 

  • Lederman, N. G. (2007). Nature of science: Past, present, and future. In S. Abell & N. Lederman (Eds.), Handbook of research on science education (pp. 831–879). Mahwah, NJ: Lawrence Erlbaum.

  • Lee, O. & Anderson, C. W. (1993). Task engagement and conceptual change in middle school science classrooms. American Educational Research Journal, 30(3), 585–610.

    Article  Google Scholar 

  • Linacre, J. M. (2002). Winsteps Manual. Chicago, IL: Winsteps.

  • Liu, X. (2007). Elementary to high school students’ growth over an academic year in understanding concepts of matter. Journal of Chemical Education, 84(11), 1853–1856.

  • Liu, X. (2010a). Essentials of science classroom assessment. London, United Kingdom: Sage Publications.

  • Liu, X. (2010b). Using and developing measurement instruments in science education: A Rasch modeling approach. Charlotte, NC: Information Age Publishing.

  • Liu, X. (2012). Using Learning Progression to Organize Learning Outcomes: Implications for Assessment. In S. Bernholt, K. Neumann, & P. Nentwig (Eds.), Making It Tangible - Learning Outcomes in Science Education (pp. 309–325). Münster, Germany: Waxmann.

  • McClary, L. M. & Bretz, S. L. (2012). Development and assessment of a diagnostic tool to identify organic chemistry students’ alternative conceptions related to acid strength. International Journal of Science Education, 34(15), 2317–2341.

    Article  Google Scholar 

  • Meluso, A., Zheng, M., Spires, H. A. & Lester, J. (2012). Enhancing 5th graders’ science content knowledge and self-efficacy through game-based learning. Computers & Education, 59(2), 497–504.

    Article  Google Scholar 

  • Mintzes, J. J., Wandersee, J. H. & Novak, J. D. (2005). Assessing science understanding: A human constructivist view. San Diego, CA: Academic.

  • National Research Council. (2012). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. Washington, DC: National Academies Press.

  • National Research Council (2013). Next generation science standards: For states, by states. Washington, DC: National Academies Press.

    Google Scholar 

  • Nunnally, J. C., Bernstein, I. H. & Berge, J. M. T. (1967). Psychometric theory (1st ed.). New York, NY: McGraw-Hill.

  • Osborne, J. & Dillon, J. (2008). Science education in Europe. A report to the Nuffield Foundation. London, United Kingdom: King’s College.

  • Rasch, G. (1993). Probabilistic models for some intelligence and attainment tests. Retrieved from ERIC database (ED419814).

  • Roediger, H. L., III & Marsh, E. J. (2005). The positive and negative consequences of multiple-choice testing. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31(5), 1155–1159.

    Google Scholar 

  • Sesli, E. & Kara, Y. (2012). Development and application of a two-tier multiple-choice diagnostic test for high school students’ understanding of cell division and reproduction. Journal of Biological Education, 46(4), 214–225.

    Article  Google Scholar 

  • Smith, C. L., Wiser, M., Anderson, C. W. & Krajcik, J. (2006). Implications of research on children’s learning for standards and assessment: A proposed learning progression for matter and the atomic-molecular theory. Measurement: Interdisciplinary Research & Perspective, 4(1–2), 1–98.

    Google Scholar 

  • Smith, A. B., Rush, R., Fallowfield, L. J., Velikova, G. & Sharpe, M. (2008). Rasch fit statistics and sample size considerations for polytomous data. BMC Medical Research Methodology, 8(1), 1–33.

    Article  Google Scholar 

  • Stoddart, T., Abrams, R., Gasper, E. & Canaday, D. (2000). Concept maps as assessment in science inquiry learning-a report of methodology. International Journal of Science Education, 22(12), 1221–1246.

    Article  Google Scholar 

  • van Driel, J. H., Meirink, J., Van Veen, K. & Zwart, R. (2012). Current trends and missing links in studies on teacher professional development in science education: A review of design features and quality of research. Studies in Science Education, 48(2), 129–160.

    Article  Google Scholar 

  • Zeidler, D. & Sadler, T. (2009). Scientific literacy, PISA, and socioscientific discourse: Assessment for progressive aims of science education. Journal of Research in Science Teaching, 46(8), 909–921.

    Article  Google Scholar 

Download references

Acknowledgements

This paper is based upon work supported by the National Science Foundation under Grant No. DUE-1102998. Any opinions, findings, and conclusions or recommendations expressed in the materials are those of the authors and do not necessarily reflect the views of the National Science Foundation.

Author information

Authors and Affiliations

  1. Learning and Instruction, University at Buffalo, SUNY, 367 Baldy Hall, Buffalo, NY, 14260-1000, USA

    Yang Yang & Xiufeng Liu

  2. Institution of Chemical Education, Northeast Normal University, 5268 Renmin Street, Changchun, 130024, People’s Republic of China

    Peng He

Authors
  1. Yang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peng He
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiufeng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yang Yang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, Y., He, P. & Liu, X. Validation of an Instrument for Measuring Students’ Understanding of Interdisciplinary Science in Grades 4-8 over Multiple Semesters: a Rasch Measurement Study. Int J of Sci and Math Educ 16, 639–654 (2018). https://doi.org/10.1007/s10763-017-9805-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10763-017-9805-7

Keywords

Search

Navigation