Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Project-based learning for middle school students monitoring standby power: replication of impact on stem knowledge and dispositions


Middle school students participating in energy-monitoring activities guided by their teachers during 2009–2011 gained (p < .05) in content knowledge and became more positive in their dispositions toward STEM (science, technology, engineering, and mathematics). No comparison group data were gathered for this initial study. Activities were replicated with a new group of treatment students during 2013–2014, adding a comparison group not receiving the treatment. Matched pre-post data from 2013 to 2014 confirmed gains in knowledge of environmental science and vampire power (p < .0001, effect size = .86). Aggregate dispositions toward science, mathematics, engineering and technology became more positive for treatment versus comparison group students (p = .023). Gains in STEM dispositions for girls were more positive (effect size = .37) than for boys. Implications of these findings are that hands-on, inquiry-based science activities may help increase the STEM career pipeline in ways that can lead to broader participation in STEM careers in the future.

This is a preview of subscription content, log in to check access.

Fig. 1


  1. 1.

    Note that average gains in content knowledge represent slightly more than one additional item correct on the post test versus the pretest, for a vampire power quiz with 10 items.


  1. Alexakos, K., & Antoine, W. (2003). The gender gap in science education: strategies to encourage female participation in science. Science Teaching,70(3), 30–33.

  2. Baker, D. (2013). What works: using curriculum and pedagogy to increase girls’ interest and participation in science. Theory into Practice,52(1), 14–20. https://doi.org/10.1080/07351690.2013.743760.

  3. Barton, A., Tan, E., & Rivet, A. (2008). Creating hybrid spaces for engaging school science among urban middle school girls. American Educational Research Journal,45, 68–103.

  4. Benson, L., & Borrego, M. (2015). The role of replication in engineering education research. Journal of Engineering Education,104(4), 388–392.

  5. Bialo, E., & Sivin-Kachala, J. (1996). The effectiveness of technology in schools: A summary of recent research. School Library Media Quarterly,25(1), 51–57.

  6. Blumenfeld, P. C., Soloway, E., Marx, R. W., Krajcik, J. S., Guzdial, M., & Palincsar, A. (1991). Motivating project-based learning: Sustaining the doing, supporting the learning. Educational Psychologist,26(3&4), 369–398.

  7. Bouvier, S. (2011). Increasing student interest in science, technology, engineering, and math (STEM): Massachusetts STEM pipeline fund programs using promising practices. Report prepared for the Massachusetts Department of Higher Education by the University of Massachusetts Donahue Institute. Retrieved from http://www.cssia.org/pdf/20000062-IncreasingStudentInterestinScience,Technology,Engineering,andMath(STEM).pdf

  8. Bybee, R. W. (2010). Advancing STEM education: A 2020 vision. Technology and Engineering Teacher,70(6), 30–35.

  9. Campbell, D. T., & Stanley, J. C. (1966). Experimental and quasi-experimental designs for research. Boston: Houghton Mifflin Company.

  10. Cavallo, A. M. L., & Laubach, T. A. (2001). Students’ science perceptions and enrollment decisions in different learning cycle classrooms. Journal of Research on Science Teaching,41(4), 392–414.

  11. Ceci, S., Williams, W., & Barnett, S. (2009). Women’s underrepresentation in science: sociocultural and biological considerations. Psychological Bulletin,135(2), 218–261. https://doi.org/10.1037/a0014412.

  12. Choi, N., & Chang, M. (2009). Performance of middle school students: Comparing U.S and Japanese inquiry-based science practices in middle schools. Middle Grades Research Journal,6(1), 15–26.

  13. Christensen, R., & Knezek, G. (2013). Contrasts in student perceptions of STEM content and careers. In R. McBride & M. Searson (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2013 (pp. 2048–2053). Chesapeake, VA: AACE. Retrieved Jan 2, 2014 fromhttp://www.editlib.org/p/48404.

  14. Christensen, R., & Knezek, G. (2015). Active learning approaches to integrating technology into middle school science classrooms: Reconceptualizing a middle school science curriculum based on 21st century skills. In X. Ge, D. Ifenthaler, & J. M. Spector (Eds.), Full steam ahead: Emerging technologies for STEAM (pp. 17–37). New York: Springer Academic.

  15. Christensen, R., & Knezek, G. (2017). Relationship of middle school student STEM interest to career intent. Journal of Education in Science, Environment and Health,3(1), 1–13.

  16. Christensen, R., Knezek, G., & Tyler-Wood, T. (2015). Gender differences in high school dispositions toward science, technology, engineering and mathematics careers. Journal of Computers in Mathematics and Science Teaching,34(4), 395–408.

  17. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.

  18. DeVellis, R. F. (1991). Scale development. Newbury Park, NJ: Sage.

  19. Earp, B. D. (2015). How to fix psychology’s replication crisis. The Chronicle of Higher Education,62, 1–9.

  20. George, R. (2006). A cross-domain analysis of change in students’ attitudes toward science and attitudes about utility of science. International Journal of Science Education,28(6), 571–589. https://doi.org/10.1080/09500690500338755.

  21. Gokhale, A. A., Rabe-Hemp, C., Woeste, L., & Machina, K. (2015). Gender differences in attitudes toward science and technology among majors. Journal of Science Education and Technology,24(4), 509–516. https://doi.org/10.1007/s10956-014-9541-5.

  22. Hammouri, H. (2004). Attitudinal and motivational variables related to mathematics achievement in Jordan: Findings from the Third International Mathematics and Science Study (TIMSS). Educational Research,46(3), 241–257. https://doi.org/10.1080/0013188042000277313.

  23. Hattie, J. (2009). Visible learning. London: Routledge.

  24. Heard, P. F., Divall, S. A., & Johnson, S. D. (2000). Can ‘ears’on’ help hands-on science learning for girls and boys? International Journal of Science Education,22, 1133–1146.

  25. Heilbronner, N. N. (2011). Stepping onto the STEM pathway. Journal for the Education of the Gifted,34(6), 876–899.

  26. Hirsch, L., Carpinelli, J., Kimmel, H., Rockland, R., & Bloom, J. (2007). The differential effects of pre-engineering curricula on middle school students’ attitudes to and knowledge of engineering careers. Presented at the 37th ASEE/IEEE Frontiers in Education Conference. Retrieved from http://fie-conference.org/fie2007/papers/1205.pdf

  27. Hodges, C. B. (2015). Replication studies in educational technology. TechTrends,59(4), 3–4.

  28. Huziak-Clark, T., Sondergeld, T., van Staaden, M., Knagga, C., & Bullerjahn, A. (2015). Assessing the impact of a research-based STEM program on STEM majors’ attitudes and beliefs. Journal of School Science and Mathematics,115(5), 226–236.

  29. Kim, H. (2016). Inquiry-based science and technology enrichment program for middle school-aged female students. Journal of Science Education and Technology,25, 174–186. https://doi.org/10.1007/s10956-015-9584-2.

  30. Knezek, G., & Christensen, R. (1998). Internal consistency reliability for the Teachers Attitudes toward Information Technology Survey. In S. McNeil, J. Price, S. Boger-Mehall, B. Robin, & J. Willis (Eds.), Proceedings of SITE 1998Society for Information Technology & Teacher Education International Conference (pp. 843–844). Chesapeake, VA: Association for the Advancement of Computing in Education (AACE).

  31. Knezek, G., Christensen, R., & Tyler-Wood, T. (2011). Contrasting perceptions of STEM content and careers. Contemporary Issues in Technology and Teacher Education,11(1), 92–117.

  32. Knezek, G., Christensen, R., Tyler-Wood, T., & Periathiruvadi, S. (2013). Impact of environmental power monitoring activities on middle school student perceptions of STEM. Science Education International,24(1), 98–123.

  33. Lenhard, W. & Lenhard, A. (2016). Calculation of effect sizes. Retrieved from: https://www.psychometrica.de/effect_size.html. Dettelbach: Psychometrica. https://doi.org/10.13140/rg.2.1.3478.4245

  34. Liu, F. (2008). Impact of online discussion on elementary teacher candidates’ anxiety towards teaching mathematics. Education,128(4), 614–629.

  35. Makel, M.C., & Plucker, J.A. (2014). Facts are more important than novelty: Replication in the education sciences. Educational Research, 20(10), 1–13. Retrieved from http://edr.sagepub.com/content/early/2014/07/23/0013189X14545513.

  36. Maltese, A.V., & Tai, R.H. (2011). Pipeline persistence: Examining the association of educational experiences with earned degrees in STEM among U.S. students. Science Education, 95(5), 877-907.

  37. McCoy, J. (2006). Improving middle school students’ attitudes towards science. Journal of Teacher Initiated Research, 3(10), 129–134. Retrieved from http://www.otterbein.edu/Education/JTIR/VolumeIII/mccoy.pdf.

  38. Meier, A. (2009). National Geographic Vampire Power Test. Retrieved from http://ngm.nationalgeographic.com/2008/04/climate/standby-quiz-interactive.

  39. Miettinen, R. (2000). The concept of experiential learning and John Dewey’s theory of reflective thought and action. International Journal of Lifelong Education,19(1), 54–72.

  40. Milman, N., Christensen, R., Spector, J.M., Branch, R., Schmidt-Crawford, D., Hodges, C., et al. (2016). A revised replication study typology and a call for participation: Replication studies involving technology and teacher education. In Proceedings of Society for Information Technology & Teacher Education International Conference 2016 (pp. 1054–1058). Chesapeake, VA.

  41. Misiti, F., Shrigley, R., & Hanson, L. (1991). Science attitude scale for middle school students. Science Education,75(5), 525–540. https://doi.org/10.1002/sce.3730750504.

  42. National Center for Education Statistics. (2012a). The Nation’s Report Card: Science, 2009 and 2011. Retrieved from http://www.nationsreportcard.gov/science_2011/g8_nat.aspx

  43. National Center for Education Statistics. (2012b). The Nation’s Report Card: Science in Action: Hands-On and Interactive Computer Tasks From the 2009 Science Assessment (NCES 2012-468). Institute of Education Sciences: Department of Education, Washington, D.C.

  44. National Science Foundation. (2014). Women, minorities, and persons with disabilities in science and engineering: Data tables. Retrieved from http://www.nsf.gov/statistics/wmpd/2013/tables.cfm.

  45. Neuhauser, A. (2014). Space Cadets: STEM Program Gives Students Control of Satellites. U.S. News & World Report. Retrieved from https://www.usnews.com/news/stem-solutions/articles/2014/08/27/space-cadets-stem-program-gives-students-control-of-satellites

  46. Nolte, P. (2014). Middle Schoolers Out to Save the World (MSOSW) External Evaluation Report. Unpublished document. Institute for the Integration of Technology into Teaching & Learning: Denton, TX.

  47. Osborne, J., Simon, S., & Collins, S. (2003). Attitudes toward science: A review of the literature and its implications. International Journal of Science Education,25(9), 1049–1079.

  48. Palmer, D. G. (2009). Student interest generated during an inquiry skills lesson. Journal of Research on Science Teaching,46(2), 147–165.

  49. Peterman, K., Kermish-Allen, R., Knezek, G., & Christensen, R. (2016). Measuring student career interest within the context of technology-enhanced STEM projects: A cross-project comparison study based on the career interest questionnaire. Journal of Science Education and Technology,25(4), 833–845. https://doi.org/10.1007/s10956-01609617-5.

  50. Pine, J., Aschbacher, P., Roth, E., Joes, M., McPhee, C., Martin, C., et al. (2006). Fifth graders’ science inquiry abilities: A comparative study of students in hands-on and textbook curricula. Journal of Research on Science Teaching,43(5), 467–484.

  51. Spector, J. M., Johnson, T. E., & Young, P. A. (2015). An editorial on replication studies and scaling up efforts. Educational Technology Research and Development,63(1), 1–4. https://doi.org/10.1007/s11423-014-9364-3.

  52. Super, D. E. (1969). Vocational development theory. The Counseling Psychologist,1(1), 2–30.

  53. Tai, R. H., Liu, C. Q., Maltese, A. V., & Fan, X. (2006). Planning early for careers in science. Science,312, 1143–1144.

  54. Tyler-Wood, T., Knezek, G., & Christensen, R. (2010). Instruments for assessing interest in STEM content and careers. Journal of Technology and Teacher Education,18(2), 341–363.

  55. Tyler-Wood, T. L., Ellison, A., Lim, O., & Periathiruvadi, S. (2011). Bringing up girls in science (BUGS): The effectiveness of an afterschool environmental science program for increasing female student’s interest in science careers. Journal of Science Education Technology.,21, 46–55. https://doi.org/10.1007/s10956-011-9279-2.

  56. U.S. Department of Energy. (2011). When to turn off personal computers. Retrieved from http://www.energysavers.gov/your_home/appliances/index.cfm/mytopic=10070

  57. Zaichkowsky, J. L. (1985). Measuring the involvement construct. Journal of Consumer Research,12(3), 341–352.

Download references


The authors would like to acknowledge the encouragement and guidance received for many years and dedicate this article in the memory of Julio Lopez-Ferrao, the NSF Program Officer for the MSOSW project.


This research was funded in part by the National Science Foundation (NSF) Innovative Technology Experiences for Students and Teachers (ITEST) Grants #0833706 and #1312168.

Author information

Correspondence to Gerald Knezek.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.



Survey items including 20 content items and STEM semantics survey


Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Knezek, G., Christensen, R. Project-based learning for middle school students monitoring standby power: replication of impact on stem knowledge and dispositions. Education Tech Research Dev 68, 137–162 (2020). https://doi.org/10.1007/s11423-019-09674-3

Download citation


  • STEM dispositions
  • STEM content knowledge
  • Energy conservation
  • Middle school students
  • Replication study