Keywords

1.1 Introduction

With the increasing emphasis on science, technology, engineering, and mathematics (STEM) education and careers, it is important to assess students throughout their education in the core subjects of mathematics and science, and identify persisting student misconceptions, errors, and misunderstandings. Understanding how misconceptions, errors, and misunderstandings in the higher grade levels relate to a lack of foundational understanding at earlier grades is important for many stakeholders in science and mathematics education, including classroom teachers, teacher educators, policymakers, and researchers. This report analyzes specific student misconceptions, errors, and misunderstandings related to core physics and mathematics concepts; the results may inform improvements in the teaching, learning, and reinforcement of these core concepts throughout elementary, middle, and secondary school.

We used assessment items and student performance data from the Trends in International Mathematics and Science Study (TIMSS) and TIMSS Advanced assessments conducted across 20 years (1995–2015)Footnote 1 to explore students’ level of understanding of two core topics (gravity and linear equations), and the nature and extent of their misconceptions, errors, and misunderstandings at grade four and grade eight (TIMSS students), and in the final year of secondary school (TIMSS Advanced). We report results for five countries that participated in the TIMSS Advanced 2015 assessment, namely Italy, Norway, the Russian Federation, Slovenia, and the United States. These countries were selected from the nine countries that participated in TIMSS Advanced 2015 as they also participated in all, or nearly all, TIMSS grade four and grade eight assessments from 1995 to 2015. The data thus maximize the number of comparisons across countries and grade levels, and enable us to report performance patterns over time across multiple assessment cycles.Footnote 2 The other four countries that participated in TIMSS Advanced 2015 (France, Lebanon, Portugal, and Sweden) did not participate in TIMSS 2015 at both grades four and eight, or had missing data for more than one prior assessment cycle for at least one grade level. The specific assessments in which each country participated are summarized in Chap. 3.

Using TIMSS and TIMSS Advanced assessment data to explore student misconceptions, errors, and misunderstandings has multiple advantages. First, the TIMSS and TIMSS Advanced assessments have been administered to nationally representative samples of students at regular intervals, starting in 1995 (with the most recent assessments conducted in 2015).Footnote 3 In contrast, most research studies investigating student misconceptions use fairly small samples from a particular region, district, or school (Alonzo et al. 2012) and are conducted within a limited time frame. Second, TIMSS provides the ability to track performance of student cohorts at three grade levels across multiple assessment years, permitting the evaluation of student performance and misconceptions over time. Lastly, TIMSS and TIMSS Advanced provide access to sets of released items (questions from the assessments) and student performance data from each assessment cycle that can be used for research purposes, such as the diagnostic item-level results in this report. The results may provide a more comprehensive picture of student performance within and across countries.

TIMSS and TIMSS Advanced data have been used in a number of secondary analyses conducted to address the topic of student misconceptions in different countries (Angell 2004; Juan et al. 2017; Mosimege et al. 2017; Prinsloo et al. 2017; Provasnik et al. 2019; Saputro et al. 2018; Văcăreţu n.d.; Yung 2006). Following the release of the 2015 TIMSS and TIMSS Advanced results in the United States (Provasnik et al. 2016), the American Institutes for Research (AIR) conducted in-depth secondary analyses of TIMSS and TIMSS Advanced data from the United States. An initial report on the United States’ performance in TIMSS Advanced 2015 described areas of relative strength and weakness, and common approaches, misconceptions, and errors in advanced mathematics and physics (Provasnik et al. 2019). A follow-up study using both TIMSS and TIMSS Advanced data further explored how physics misconceptions demonstrated by TIMSS Advanced students in the United States can be traced back to misconceptions, or a lack of foundational understanding about physics concepts in earlier grades (unpublished work).Footnote 4

In this report, we expand upon such previous work and describe the methodology we use to (1) investigate misconceptions, errors, and misunderstandings in both physics and mathematics; (2) explore patterns of misconceptions, errors, and misunderstandings across grade levels for a select group of countries; (3) report differences in these patterns across countries, overall and by gender; and (4) report differences across assessment years.

1.2 Defining the Terminology

To begin, we first define the terms used throughout this report as they apply to physics and mathematics.Footnote 5

1.2.1 Performance Objectives

Performance objectives are based on the set of TIMSS and TIMSS Advanced items selected for the study. They describe the specific knowledge and abilities expected of students at different grade levels (i.e., what they must know and be able to do in order to respond correctly to the TIMSS and TIMSS Advanced assessment items). For this report, there are four performance objectives identified related to gravity and nine related to linear equations, each measured by one or more assessment items (see Chap. 4). Some performance objectives were assessed at only one grade level, while others were assessed by items in two grade levels (i.e., TIMSS Advanced/grade eight or grade eight/grade four) or in all three grade levels (for physics only).

1.2.2 Misconceptions in Physics

Misconceptions apply only to the physics items. These reflect students’ incorrect preconceived notions about a physics concept, usually based on their experiences or observations of physical phenomena in daily life. In this report, a misconception is demonstrated by particular types of student responses such as specific incorrect response options for multiple-choice items or specific incorrect scoring guide categories for constructed-response items (where students provide a written response).

1.2.3 Errors in Mathematics

Errors apply only to mathematics items where students are expected to follow a set mathematical procedure to obtain the correct response. Errors reflect any type of response where the correct answer was not obtained.

1.2.4 Misunderstandings in Physics and Mathematics

Misunderstandings can apply to both physics and mathematics items. These reflect responses where students did not demonstrate that they understood the physics or mathematics concept as it applies to the item, but do not involve procedural errors in mathematics or signify a specific misconception in physics as defined above.

  • Physics Items

    Includes items (mostly constructed-response) where students must apply their understanding of the physics concept to a given situation, but specific incorrect response types are not tracked. Misunderstandings in physics indicate a lack of understanding and include all incorrect responses (including off-task and blank responses).

  • Mathematics Items

    Includes items where there is no set procedure required, and students must figure out how to apply their understanding of the mathematics concept to answer the question. A misunderstanding in mathematics may be demonstrated by specific types of incorrect student responses or by all incorrect responses (including off-task and blank responses).

1.3 Core Concepts in Physics and Mathematics

We focus on core concepts in physics and mathematics that are introduced in elementary school and further developed across grades through middle school and secondary school. To fully demonstrate our methodology for exploring students’ misconceptions, errors, and misunderstandings across grade levels, we selected the specific topics of gravity in physics and linear equations in mathematics. These topics reflect key concepts that are covered in both the TIMSS and TIMSS Advanced assessment frameworks, and there are items covering these topics (or their precursors) in the grade four and eight assessments, and the TIMSS Advanced assessment. This allowed us to trace misconceptions, errors, and misunderstandings across all three grade levels.

Gravity is a fundamental concept introduced to students at an early age, and students enter school with preconceptions about the topic based on their experiences and observations of physical phenomena in their daily life. The topic is covered in physical science, earth science, and more advanced physics courses in secondary school, and the depth of understanding related to gravity is expected to develop across the grades. The topic of gravity (gravitational force) provides a good context for evaluating students’ abilities to apply force concepts, and can be used to identify some general misconceptions related to force and motion across all three grade levels.

Based on the TIMSS 2015 frameworks (Jones et al. 2013), students at grade four can identify gravity as the force that draws objects toward Earth and recognize that forces may cause an object to change its motion (Table 1.1). At grade eight, students can describe common mechanical forces, including gravitational force, acting on objects and can predict the changes in motion (if any) of an object based on the forces acting on it. In addition, by grade eight, students recognize that it is the force of gravity that keeps planets and moons in orbit and pulls objects to Earth’s surface. The 2015 TIMSS Advanced physics framework (Jones et al. 2014) expects students at the end of secondary school to use Newton’s laws of motion to explain the dynamics of different types of motion and how the action of combined forces influences a body’s motion.

Table 1.1 TIMSS 2015 and TIMSS Advanced 2015 assessment framework objectives related to gravity
Full size table

For mathematics, we focused on linear equations for several reasons. Algebra, and the topic of linear equations specifically, spans students’ mathematics education in elementary, middle school, and secondary school. In the 2015 TIMSS mathematics framework (Grønmo et al. 2013), students at grade four can identify or write expressions or number sentences to represent problem situations involving unknowns; identify and use relationships in well-defined patterns; solve problems set in contexts; and read, compare, and represent data from tables and line graphs (Table 1.2). At grade eight, students can write equations or inequalities to represent situations; solve simultaneous linear equations in two variables; interpret, relate, and generate representations of functions in tables, graphs, or words; and interpret the meanings of slope and y-intercept in linear functions. The 2015 TIMSS Advanced mathematics framework (Grønmo et al. 2014) expects students at the end of secondary school to solve linear and quadratic equations, as well as systems of linear equations and inequalities, and to use equations and inequalities to solve contextual problems.

Table 1.2 TIMSS 2015 and TIMSS Advanced 2015 assessment framework objectives related to linear equations: 2015
Full size table

Not only do students continue to study the topic of linear equations across grades, their conceptual understanding of linear equations progresses from concrete (number sentences at grade four) to abstract (equations and graphical representations at grade eight and the upper secondary level) as their mathematics competency progresses. In addition, students’ performance in algebra is linked to higher achievement in mathematics (Walston and McCarroll 2010). The topic of linear equations is one of the most basic in algebra because linear equations are much simpler than other types of relationships, such as quadratic and exponential equations. Before students can understand characteristics like intercepts and slope in these more complex relationships, they must master the same characteristics in linear equations. Finally, the topic of linear equations is versatile in terms of connecting mathematics to other subject areas and real-world applications. For example, understanding graphs of equations is an integral skill in science classrooms. Similarly, understanding equations is important in general life skills, including all aspects of financial literacy. The focus on linear equations will, therefore, provide an examination of students’ performance in a topic area that is important for postsecondary success.

1.4 Research Questions

Our methodology (see Chap. 3) includes three major components: (1) assessment framework review and content mapping to identify the set of items measuring the topics of interest at each grade level; (2) evaluation of diagnostic item-level performance data to identify the specific performance objectives measured by these items and to provide evidence of specific misconceptions, errors, and misunderstandings; and (3) analyses of the percentage of students demonstrating these misconceptions, errors, and misunderstandings across countries by grade level, gender, and assessment year. Example items are shown in the report to illustrate the specific types of misconceptions, errors, and misunderstandings demonstrated by students at each grade level.Footnote 6

Using item-level performance data from multiple assessment cycles of TIMSS and TIMSS Advanced (from 1995 to 2015), we addressed three research questions.

  • Research question 1: What are common types of student misconceptions, errors, and misunderstandings in grade four, grade eight, and the final year of secondary school (TIMSS Advanced students), and how do they compare across countries?

For each selected country, we determined the frequency of specific types of misconceptions, errors, and misunderstandings related to gravity and linear equations demonstrated on items from across the three grade levels, and identified and compared patterns across countries and grade levels.

  • Research question 2: How do student misconceptions, errors, and misunderstandings differ by gender?

For each assessment item, we determined differences in student performance and the frequency of specific types of misconceptions, errors, and misunderstandings by gender, and compared these differences across countries and grade levels.

  • Research question 3: How persistent are patterns in misconceptions, errors, and misunderstandings over time?

Using trend items administered in multiple assessment cycles, we compared the frequency of specific types of misconceptions, errors, and misunderstandings across all of the TIMSS assessments conducted between 1995 and 2015 to discover whether patterns across countries changed over time (e.g., did specific misconceptions at grades four or eight increase, decrease, or stay the same between 2007 and 2015?).Footnote 7

While this report is focused on student misconceptions, errors, and misunderstandings related to the two topics of gravity and linear equations, the general methodology described in the report can be applied to a range of mathematics and science topics covered in TIMSS and TIMSS Advanced. This methodology can be used to trace misconceptions across all three grade levels (as in this report) or two grade levels (e.g., grade eight and grade four), or it can be used to focus on patterns of misconceptions at one grade only. The results can inform instruction across grades, by relating country-level patterns in misconceptions, errors, and misunderstandings to specific gaps or deficiencies in the curricula.