# Results for Student Misconceptions, Errors, and Misunderstandings in Physics and Mathematics

## Abstract

Diagnostic item-level student performance data from twenty years of TIMSS and TIMSS Advanced assessments can be used to explore students’ level of understanding of gravity and linear equations across grades four, eight, and the final year of secondary school (TIMSS Advanced students). Sets of assessment items at each grade level illustrate the nature and extent of student misconceptions, errors, and misunderstandings across grade levels in five countries (Italy, Norway, the Russian Federation, Slovenia, and the United States). The results include assessment of how students in each country performed on the set of items measuring understanding of the physics and mathematics concepts explored in this study (gravity and linear equations); common types of student misconceptions, errors, and misconceptions across grade levels in each country; patterns in misconceptions, errors, and misunderstandings across countries; and gender differences in the frequency of misconceptions, errors, and misunderstandings in each country. The frequency of specific types of student misconceptions, errors, and misunderstandings at each grade level varied across the five countries. Gender differences were found at all three grade levels, but were more extensive for physics than mathematics. Trend items administered in multiple assessment years indicated that the frequency of certain student misconceptions, errors, and misunderstandings decreased over time, while the frequency of others increased.

## Keywords

Diagnostic data Errors Gender differences Gravity International large-scale assessment Item-level data Linear equations Mathematics Misconceptions Performance objectives Physics Science Student achievement Trend analysis Trends in International Mathematics and Science Study (TIMSS) Italy Norway Russian Federation Slovenia United States## 4.1 Introduction

The results for physics (Sect. 4.2) and mathematics (Sect. 4.3) start with an overview of the set of TIMSS and TIMSS Advanced items that measure student understanding of the key concepts that are the focus of this study (gravity in physics and linear equations in mathematics). The overview also describes the performance objectives that are assessed by the items across grade levels and the types of student misconceptions, errors, and misunderstandings demonstrated on these items.^{1}

We present released example items from TIMSS and TIMSS Advanced to demonstrate specific types of student misconceptions, errors, and misunderstandings along with tabular data showing the percentage of students for each response type by country and on average across the five countries. All example items shown in the report are the standard international version.^{2}

In addition to example items, the results include tables and figures that show patterns in the percentage of students demonstrating specific misconceptions, errors, and misunderstandings across countries and grades based on the set of items that measure them.^{3} Tree graphs show the female–male difference across countries at each grade level.^{4} At grade four and grade eight, separate trend graphs for each country show the percentage of students with misconceptions, misunderstandings, and errors over multiple assessment years.^{5}

With the exception of the trend item results, all data reflect the most recent assessment in which each item was administered from 1995 to 2015, which for examples items, is the year the item was released (supplementary materials providing standard errors for all estimates are available for download at www.iea.nl/publications/RfEVol9).

## 4.2 Physics Results

Physics performance objectives related to gravity with related misconceptions and misunderstandings, by grade level

Performance objective | Related misconceptions and/or misunderstandings | Grade level | ||
---|---|---|---|---|

TA | G8 | G4 | ||

PO1: Determine the acceleration of thrown objects (after they are released) | Gravitational force (acceleration) acting on objects near Earth’s surface is not constant but changes with the height of the object above the surface. (P1A) | ✓ | ||

Objects thrown upward have no acceleration at their maximum height where the instantaneous velocity is zero (the instant it stops moving upward and reverses direction). (P1B) | ✓ | |||

Gravitational acceleration is always in the direction of motion/velocity (rather than a constant acceleration directed toward the center of Earth). (P1C) | ✓ | |||

PO2: Determine the time duration between different points on the path of a thrown object | The time on the way up and the time on the way down are not equal (the downward acceleration due to gravity is not treated as constant). (P2) | ✓ | ||

PO3: Determine the effect of gravitational force on moving objects or on objects at rest | Gravity acts only on falling objects, but not on objects at rest (on the ground or sitting on another surface) or on objects that are moving upward. (P3A) | ✓ | ✓ | |

Gravity alone cannot cause an object initially at rest to start moving; it requires another force/push. (P3B) | ✓ | ✓ | ||

PO4: Identify the direction of the force due to gravity | Gravitational force causes objects to fall “down” (in an “absolute downward” direction in space) rather than toward the center of Earth. (P4A) | ✓ | ||

Gravity pushes upward on objects sitting on a solid surface and on objects that are moving upward. (P4B) | ✓ | |||

Gravity can move objects in other directions that are not “down” toward the surface of Earth. (P4C) | ✓ |

List of TIMSS and TIMSS Advanced physics items related to gravity, organized by performance objective and grade level

TIMSS item | Grade level | Assessment year(s) | Item format | Item description | Figure in this report | Physics misconceptions and misunderstandings | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

P1A | P1B | P1C | P2 | P3A | P3B | P4A | P4B | P4C | ||||||

| ||||||||||||||

Item 1A (PA33061A) | TIMSS Advanced | 2015 | MC | Motion of a ball thrown upward—acceleration at highest point | 4.4 | ✓ | ✓ | |||||||

Item 2 (PA13063) | 1995 | CR | Acceleration of a bouncing ball | 4.5 | ✓ | |||||||||

| ||||||||||||||

Item 1B (PA33061B) | TIMSS Advanced | 2015 | CR | Motion of a ball thrown upward—time between two points | 4.4 | ✓ | ||||||||

| ||||||||||||||

Item 3 (PA23014) | TIMSS Advanced | 2008 | CR | Forces acting on a stone thrown upward | 4.6 | ✓ | ||||||||

Item 4 (S032141) | Grade 8 | 2003, 2007, 2011 | MC | Gravity acting on parachute jumper | 4.7 | ✓ | ||||||||

Item 5 (S022012) | 1999 | MC | Gravity acting on a rocket being launched from Earth | – | ✓ | |||||||||

Item 6 (S012075) | 1995 | MC | Gravity acting on an apple falling from a tree | – | ✓ | |||||||||

Item 7 (S042211) | 2007, 2011, 2015 | CR | Forces acting on people sitting on a wall | 4.8 | ✓ | |||||||||

Item 8 (S032281) | 2003 | MC | Why helium balloon moves upward | 4.9 | ✓ | |||||||||

Item 9 (S042293A) | 2007, 2011, 2015 | CR | Force causing a ball thrown upward to fall | 4.10 | ✓ | |||||||||

| ||||||||||||||

Item 10 (S031311) | Grade 4 | 2003, 2007, 2011 | MC | Force causing an object to fall to the ground | 4.12 | ✓ | ||||||||

Item 11 (S051147) | 2011, 2015 | CR | Force causing a marble to roll down a sloping track | 4.13 | ✓ | |||||||||

| ||||||||||||||

Item 12 (S032714) | Grade 8 | 2003 | MC | Direction gravity makes a ball fall at different places on Earth | 4.11 | ✓ | ||||||||

Item 13 (S041119) | Grade 4 | 2007, 2011, 2015 | MC | Direction of the force of Earth’s gravity | 4.14 | ✓ | ||||||||

Item 14 (S061048) | 2015 | MC | Direction of movement due to gravity | – | ✓ | ✓ | ||||||||

Item 15 (S041308) | 2007 | MC | Example of an object moving due to gravity | 4.15 | ✓ | |||||||||

Item 16 (S031313) | 2003 | MC | Force that makes objects repel each other | 4.16 | ✓ |

### 4.2.1 Student Performance on TIMSS and TIMSS Advanced Items Related to Gravity

^{6}Average item performance across the five countries for TIMSS Advanced ranged from 25 to 54% correct, compared with a range of 36–79% correct at grade eight and 31–78% correct at grade four. Based on the average performance, the range of item percent correct was lower in TIMSS Advanced (25–42% correct) than at grade eight (36–79% correct) or grade four (31–78% correct). Across the set of gravity items at each grade level, at least half of students provided a correct response on five out of seven grade eight items and three out of six grade four items, compared to only one out of the four TIMSS Advanced items. However, there were notable differences in performance patterns observed across the five countries.

Across the four TIMSS Advanced items (Fig. 4.1), the broadest range of item performance was in Slovenia (from 16 to 72% correct) and the United States (from 15 to 64% correct). In contrast, item-level performance ranged from 23 to 49% correct in the Russian Federation and, in Norway, was clustered between 46 and 63% correct. Performance was lowest in Italy, with the item percent correct ranging from 9 to 31%, and generally higher in Norway when considering the full set of items. The most difficult item in all countries was item 2 (“acceleration of a bouncing ball”), a CR item under performance objective 1 (“determine the acceleration due to gravity of thrown objects (after they are thrown)”).^{7} In comparison, item 1B (“motion of a ball thrown upward–time between two points”), a CR item from performance objective 2 (“determine the time duration between different points on the path of a thrown object”), was the easiest item for all countries except Italy, with at least 60% of students correct in Norway (63%), the United States (64%), and Slovenia (72%).

At grade eight (Fig. 4.2), a broad range of item performance was found in all five countries and especially in Norway (from 32 to 85% correct). Three MC items, namely items 4, 5, and 6 from performance objective 3 (“determine the effect of gravitational force on moving objects or on objects at rest”), were among the most difficult items in all countries. In particular, performance on item 4 (“gravity acting on a parachute jumper”) ranged from 26% correct in Italy to 30–40% correct in Norway, the Russian Federation, and the United States, to 47% correct in Slovenia. In comparison, the easiest item was item 12 (“direction gravity makes a ball fall at different places on Earth”) from performance objective 4 (“identify the direction of the force due to gravity”), with at least 70% correct in all five countries, and more than 80% correct in Norway, the Russian Federation, and Slovenia.

The broadest range of item performance was found at grade four (Fig. 4.3) in Italy, the Russian Federation, and Slovenia, compared to Norway (where the range was greatest at grade eight) and the United States (where the range was more similar across the three grade levels). In particular, item performance in Slovenia ranged from 16 to 88% correct (a spread of 72 percentage points at grade four). Item 11 from performance objective 3 (“determine the effect of gravitational force on moving objects or on objects at rest”) and items 14 and 16 from performance objective 4 (“identify the direction of the force due to gravity”), were among the most difficult items in all countries, although the specific pattern of performance varied. The CR item 11 (“force causing a marble to roll down a sloping track”) ranged from <20% correct in Italy (15%) and Slovenia (16%), to 29% correct in Norway, to ≥40% correct in the Russian Federation (53%) and the United States (40%). Performance on the MC item 14 (“direction of movement due to gravity”) ranged from 32% correct in Norway to 68% correct in the United States, and performance on MC item 16 (“force that makes objects repel each other”) ranged from 26% correct in Slovenia to 45% correct in the United States. In comparison, item 10 (“force causing an object to fall to the ground”) from performance objective 3 and item 13 (“direction of the force of Earth’s gravity”) from performance objective 4 (both MC items) were amongst the easiest items across all countries, with ≥60% of students responding correctly to both items (except in Slovenia, with 57% correct on item 10). Performance on item 10 was particularly high in the United States (82% correct) and performance on item 13 was particularly high in the Russian Federation (95% correct) and Slovenia (88% correct).

### 4.2.2 Common Types of Misconceptions and Misunderstandings Related to Gravity Across Countries

^{−2}) directed toward the center of Earth. By the end of secondary school, students are expected to understand that the only forces acting on a thrown object (after it is released) are the downward force due to gravity and air resistance and that the observed motion (slowing, reaching a maximum height, and then falling back down) is the result of the constant acceleration due to gravity at all positions in the path of the object. The first example item (item 1: Fig. 4.4, and Tables 4.3 and 4.4) requires students to apply Newton’s laws of motion to answer two questions about the motion of a ball thrown vertically upward, and shows that many TIMSS Advanced students had difficulty applying these concepts.

Student performance data for physics item 1A (PA33061A), 2015

Student performance data for physics item 1B (PA33061B), 2015

Part A is a MC item requiring students to identify the acceleration of the ball at its highest position (the instant it stops moving upward and reverses direction). A correct response to part A (option D) requires students to know that the acceleration due to gravity is constant and applies equally to the ball at all positions. Across the five countries included in the study, the percent correct ranged from 9% (Italy) to 56% (Norway), with an international average of 42% correct. On average, nearly half of students internationally (48%) indicated that the acceleration was zero (option A), demonstrating the misconception that there is no acceleration since the instantaneous velocity at that position is zero (rather than a constant acceleration due to gravity at all positions). This misconception was less common in Norway (39%), but this still reflects more than one-third of students.

Another 8% of students on average across countries incorrectly determined that the magnitude of the acceleration due to gravity was different at point 3 (either half or twice that at point 2), demonstrating the misconception that the force of gravity changed with the height of the ball (options B and C). Students selecting these options may be confusing gravitational force with gravitational potential energy (*PE*_{g} = *mgh*), which increases with height (*h*), or incorrectly applying Newton’s law of universal gravitation (*F*_{g} = *Gm*_{1}*m*_{2}*/r*^{2}), where gravitational force (*F*_{g}) decreases with the distance squared (*r*^{2}). Although this law can be applied to objects that are far from Earth, the difference in force or acceleration due to a change in height is negligible for objects near Earth’s surface, where the acceleration due to gravity (*g*) is treated as a constant.

In part B, students were asked to determine the time duration between two points on the path of the ball (halfway up and halfway down). A correct response to part B requires students to indicate that the time traveled by the ball is the same on the way up as it is on the way down, as shown in the scoring guide. This relationship can be determined by applying Newton’s laws of motion to the situation where there is a constant acceleration due to gravity (*g*), and TIMSS Advanced students are expected to have covered this in their physics courses. A common misconception, though, is that the time on the way down is shorter because the ball is accelerating (speeding up) on the way down and decelerating (slowing down) on the way up (i.e., the downward acceleration due to gravity is not treated as constant). On average, slightly more than half (54%) of TIMSS Advanced students answered part B correctly. Most students in Norway (63%), Slovenia (72%), and the United States (64%) provided a correct response. In comparison, about half of the TIMSS Advanced students in the Russian Federation (49%) and about one-fifth in Italy (20%) did so.

^{8}On average, about one-quarter of students (27%) demonstrated the common misconception that the acceleration is in the same direction as the motion of the ball (along the curved path at points P and Q) and that there is no acceleration when the ball is at its maximum height (point R) (code 72 in the scoring guide). This misconception was most common in the United States (38%) and least common in Norway (14%). Another 12% internationally indicated that the acceleration of the ball is upward on its way up (point Q) and downward on its way down (point P), with either an upward or downward acceleration or no acceleration at point R (codes 70 and 71). In addition, about 8% internationally indicated that the acceleration is perpendicular to the direction of motion at points P and Q (code 74). This type of response reflects a misunderstanding that the acceleration of the ball moving along a curved path is caused by a centripetal force directed toward its center (like objects orbiting the Earth). The frequency of this misconception ranged from 1% of students in the United States to 17% of students in the Russian Federation. Another 24% of students on average provided other types of incorrect responses (codes 73 and 79), and about 4% left the item blank.

Student performance data for physics item 2 (PA13063), 1995

^{9}The percentage of students who did not identify gravity ranged from almost 3% in Slovenia

^{10}to 42% in the Russian Federation. These percentages include incorrect responses (codes 71 and 79), as well as students who left the item blank.

Student performance data for physics item 3 (PA23014), 2008

^{11}: a rocket being launched from Earth (item 5); and an apple falling from a tree (item 6). In the context of a rocket launch (item 5), about one-third of grade eight students on average (36%)

^{12}indicated that gravity acts on the rocket only when it is falling back to Earth and not when it is sitting on the launch pad or moving upward after being launched. In the context of an apple falling from a tree (item 6), 40% of students on average

^{13}indicated that gravity acts on the apple only while it is falling or still hanging from the tree, and not once it lands on the ground.

Student performance data for physics item 4 (S032141), 2011

Student performance data for physics item 7 (S042211), 2015

Student performance data for physics item 8 (S032281), 2003

Student performance data for physics item 9 (S042293A), 2015

Student performance data for physics item 12 (S032714), 2003

Student performance data for physics item 10 (S031311), 2011

Student performance data for physics item 11 (S051147), 2015

^{14}Both items 13 and 14 demonstrate that some grade four students have the misconception that gravity pushes upward on objects sitting on a solid surface or on objects that are moving upward.

Student performance data for physics item 13 (S041119**)**, 2015

^{15}27% of students on average indicated that horizontal movement of objects was due to gravity.

Student performance data for physics item 15 (S041308), 2007

Student performance data for physics item 16 (S031313), 2003

### 4.2.3 Patterns in Misconceptions and Misunderstandings Related to Gravity Across Grade Levels and Countries

^{16}

All three of the TIMSS Advanced items were related to the motion of objects thrown upward or bouncing after being thrown. Both item 1A_V1 (“motion of a ball thrown upward—acceleration at highest point,” Fig. 4.4) and item 2 (“acceleration of a bouncing ball,” Fig. 4.5) measure performance objective 1: “determine the acceleration of thrown objects (after they are released).” Based on the frequency of specific response types (Fig. 4.17), many students in all five countries demonstrated the related misconceptions that “objects thrown upward have no acceleration at their maximum height where the instantaneous velocity is zero” (P1B) and that “gravitational acceleration is always in the direction of motion/velocity” (P1C). The other misconception measured by item 1A (V2, options B and C) was “gravitational force (acceleration) acting on objects near Earth’s surface is not constant but changes with the height of the objects above the surface” (P1A). This was far less common, as many more students selected option A (acceleration is zero when the ball is at its maximum height). Across items, the percentage of students demonstrating these types of misconceptions was higher in Italy and lower in Norway than the average across countries.

Performance objective 2 (“determine the time duration between different points on the path of a thrown object”) was measured by item 1B (“motion of a ball thrown upward—time between two points”), and the misconception demonstrated (P2) was quite common in all countries. Item 3 (“forces acting on a stone thrown upward,” Fig. 4.6) measures performance objective 3: “determine the effect of gravitational force on moving objects or on objects at rest.” An incorrect response to this item illustrated a lack of understanding of how the force of gravity acts on objects when they are moving upward, which is related to misconception P3A. This misconception was more common than average in Italy and the Russian Federation and was very infrequent in Slovenia (3% of students). TIMSS Advanced students not identifying gravity in this item may have misconceptions commonly found at the lower grade levels that gravity does not act on objects while they are at rest or moving upward (see Figs. 4.18 and 4.19 for misconceptions about the effect of gravitational force in grade eight and grade four).

Considering the set of TIMSS Advanced items, we noted that the prevalence of specific types of misconceptions differed across countries. In Italy, Norway, and the Russian Federation, misconceptions P1B and P2 were more common, while in the United States, misconception P1C was most common; in Slovenia, misconceptions P1B and P1C were equally common.

The misconceptions held by TIMSS Advanced students that acceleration due to gravity is not constant can arise from related misconceptions about the force of gravity in earlier years. At grade eight (Fig. 4.18), six items measured performance objective 3: “determine the effect of gravitational force on moving objects or on objects at rest.” Five of these items, namely item 4 (Fig. 4.7), item 5 (not shown), item 6 (not shown), item 7 (Fig. 4.8), and item 8 (Fig. 4.9), measured the misconception that “gravity acts only on falling objects, but not on objects at rest or on objects that are moving upward” (P3A). This misconception was very common across countries and, in particular, on item 4 (“gravity acting on a parachute jumper”), where >50% of all students demonstrated the misconception in every country except Slovenia (where this was 46%). The same misconception was measured in the similar item 5 (“gravity acting on a rocket being launched from Earth”) and item 6 (“gravity acting on an apple falling from a tree”). Although the misconception did not appear to be as frequent on these two other items, it was still quite common in all countries, ranging from 31 to 45% on item 5 and from 28 to 48% on item 6. The misconception was also somewhat less common in item 7 (“forces acting on people sitting on a wall”), ranging from 26 to 55% of students. Only 6 to 19% of students demonstrated the misconception that “gravity does not act on objects that are moving upward” (P3A) in item 8 (“why helium balloon moves upward”). The different response patterns across these items in each country may be related to students’ familiarity with the specific contexts. In general, the frequency of misconception P3A at grade eight was less in Slovenia than on average across countries, which was also the case for the TIMSS Advanced item 3, which measured the same type of misconception (Fig. 4.17).

Grade eight item 9 (“force causing a ball thrown upward to fall,” Fig. 4.10), measured the misconception that “gravity alone cannot cause an object initially at rest to start moving; it requires another force/push” (P3B). This misconception, which was common at grade four (Fig. 4.19), was also demonstrated by many grade eight students for this item (Fig. 4.18), with the frequency ranging from 22% of students in the United States to 42% of students in Italy. On average, 30% of grade eight students demonstrated this misconception. This is similar to the average percentage of TIMSS Advanced students on item 3 (25%) (Fig. 4.17) who demonstrated the related misconception P3A by not identifying gravity as a force that acts on a stone after it is thrown straight up in the air. However, there was less variation across countries in the percentage of grade eight students demonstrating the misconception than in the percentage of TIMSS Advanced students.

The last grade eight item, item 12 (“direction gravity makes a ball fall at different places on Earth,” Fig. 4.11), measured performance objective 4: “identify the direction of the force due to gravity.” The misconception demonstrated on this item that “gravitational force causes objects to fall down (in an absolute downward direction in space) rather than toward the center of Earth” (P4A) was less common than the other two types of misconceptions demonstrated by students at grade eight. Misconceptions or misunderstandings related to the direction of the force of gravity, however, were quite common at grade four (Fig. 4.19).

At grade four, two items (10 and 11) measure performance objective 3: “determine the effect of gravitational force on moving objects or on objects at rest.” In both items, incorrect responses demonstrated the misconception that “gravity alone cannot cause an object initially at rest to start moving; it requires another force/push” (P3B), but the frequency of the misconception is quite different on the two items. The misconception was demonstrated by 15% of students on average on MC item 10 (“force causing an object to fall to the ground,” Fig. 4.12), compared to 69% on average on CR item 11 (“force causing a marble to roll down a sloping track,” Fig. 4.13). This illustrates that while most students at grade four demonstrated basic knowledge about the role of gravity in falling objects (item 10), many could not apply this in a less familiar context by connecting gravity to the motion of an object rolling down a sloped surface (item 11).

The remaining grade four items measure performance objective 4: “identify the direction of the force due to gravity.” Item 13 (“direction of the force of Earth’s gravity,” Fig. 4.14) and item 14_V1 (“direction of movement due to gravity,” not shown) measure the misconception that “gravity pushes upward on objects sitting on a solid surface and on objects that are moving upward” (P4B). This misconception was most common in Norway and Italy, and least common in the Russian Federation. Other grade four items measure the related misconception that “gravity can move objects in other directions that are not ‘down’ toward the surface of Earth” (P4C). Item 14_V2 (“direction of movement due to gravity”), item 15 (“example of an object moving due to gravity,” Fig. 4.15), and item 16 (“force that makes objects repel each other,” Fig. 4.16) all measured the misconception that gravity can make objects move in a horizontal direction, and demonstrated a lack of understanding among grade four students that gravity is an attractive force that pulls objects toward the surface of the Earth. The misconceptions on these items were generally common, ranging from 27 to 49% on average. Like misconception P4B, misconception P4C was most common in Norway. This lack of basic understanding at grade four can lead to misconceptions and misunderstandings at higher grade levels, such as those illustrated by items 7 and 12 at grade eight (Fig. 4.18) and by item 2 in TIMSS Advanced (Fig. 4.17).

Across the set of items at grade four, gravity misconceptions were frequently less common than average in the Russian Federation and the United States, and more common than average in Italy, Norway, and Slovenia. This pattern, however, did not persist across the misconceptions at higher grade levels. In particular, the percentages of TIMSS Advanced students demonstrating the gravity misconceptions were generally lower in Norway and higher in the Russian Federation (or not measurably different from the average across countries). The patterns at grade eight were more mixed, with some countries having a higher frequency of some misconceptions and a lower frequency of others. The percentage of students demonstrating gravity misconceptions in Italy was higher than the average for all countries at all three grade levels that were assessed by TIMSS.

### 4.2.4 Gender Differences in Misconceptions and Misunderstandings Related to Gravity

^{17}The only item where there were no significant gender differences in the percentage of students who were correct in any country was grade eight item 12 (“direction gravity makes a ball fall at different places on Earth”). Patterns in performance by gender differed across countries and grades. Gender differences in the percent correct were greatest on the TIMSS Advanced items, with an average male–female difference of at least 10%. In comparison, the average male–female difference in the percent correct ranged from 6 to 12% at grade eight and from 2 to 13% at grade four. In TIMSS Advanced, gender differences were pervasive in Norway, with a significantly higher percentage of males than females responding correctly to all items. In contrast, in Norway, none of the grade eight items, and three of seven items at grade four showed significant gender differences. By comparison, in the United States, there were significant gender differences on all of the grade four items, three of seven items at grade eight, and two of three items in TIMSS Advanced. In Italy, the Russian Federation, and Slovenia, significant gender differences were found for one to three items at each grade level. The specific set of items with significant gender differences varied across countries. None of the items demonstrated significant gender differences in all five countries, but there were two items at each grade level with significant gender differences in three or four countries.

Performance of female and male students on TIMSS and TIMSS Advanced physics items, by country and grade level, 1995–2015

To further understand these gender differences in item performance within and across countries, we compared the percentage of male and female students demonstrating specific types of misconceptions and misunderstandings at each grade level (Tables 4.18, 4.19, and 4.20, and Figs. 4.20, 4.21, and 4.22).^{18}

Percentage of female and male TIMSS Advanced students with misconceptions and misunderstandings about gravity, by country, 1995, 2008, and 2015

Percentage of female and male grade eight students with misconceptions and misunderstandings about gravity, by country, 1995, 1999, 2003, 2011, and 2015

Percentage of female and male grade four students with misconceptions and misunderstandings about gravity, by country: 2003, 2007, 2011, and 2015

Compared to TIMSS Advanced, there were fewer significant differences in the percentages of male and female students demonstrating the misconceptions at grade eight (Table 4.19 and Fig. 4.21). Across the five items measuring misconception P3A (“gravity acts only on falling objects, but not on objects at rest or on objects that are moving upward”), there were two items in Italy and the Russian Federation, and only one item in the United States, where a significantly higher percentage of females demonstrated the misconception. In Norway and Slovenia, none of these items showed significant gender differences. The specific set of items with a higher percentage of female students having the misconception varied across countries. On item 4 (“gravity acting on a parachute jumper”), 10% more females than males demonstrated the misconception in both Italy and the Russian Federation, while on item 5 (“gravity acting on a rocket being launched from Earth”), only the Russian Federation showed a gender difference (13% more female than male students demonstrating the misconception). There were no significant gender differences on item 6 (“gravity acting on an apple falling from a tree”). The largest gender difference was seen in Italy for item 7 (“forces acting on people sitting on a wall”), where 21% more female than male students demonstrated the misconception. In the United States, the only significant gender difference was on item 8 (“why helium balloon moves upward”), where 8% more female than male students demonstrated the misconception. The only significant gender difference on item 9 (“force causing a ball thrown upward to fall”) was in Slovenia, where 9% more females than males demonstrated misconception P3B (“gravity alone cannot cause an object initially at rest to start moving; it requires another force/push”). There were no significant differences in the percentages of male and female students demonstrating misconception P4A on item 12 (“direction gravity makes a ball fall at different places on Earth”).

As in grade eight and TIMSS Advanced, there were different patterns of gender differences in misconceptions and misunderstandings across countries on the grade four items (Table 4.20 and Fig. 4.22). Most notably, only in the United States was the frequency of misconceptions and misunderstandings significantly higher for females than males on all items, with the differences ranging from 5 to 11%. Across the two items measuring misconception P3B, significantly more females than males in Italy (11% on item 10), Norway (11% on item 11), and the United States (5% on item 10 and 11% on item 11) demonstrated the misconception that “gravity alone cannot cause an object initially at rest to start moving; it requires another force/push.” On both items measuring misconception P4B (“gravity pushes upward on objects sitting on a solid surface and objects that are moving upward”), only the United States had significantly higher percentages of females demonstrating the misconception (7–8%). The largest gender differences were found on item 14_V2 (“direction of movement due to gravity”), where the percentage of females was significantly higher than the percentage of males demonstrating misconception P4C (“gravity can make objects move in other directions that are not down toward the surface of the Earth”) in Norway (18%), the Russian Federation (9%), Slovenia (17%), and the United States (7%).

### 4.2.5 Patterns in Misconceptions and Misunderstandings Related to Gravity Over Time

At grade eight (Fig. 4.23), the trend item data covered assessment years 2003, 2007, 2011, and 2015. There were some significant differences across assessment years in the percentage of students in each country demonstrating the specific types of misconceptions. Item 4 (administered in 2003, 2007, and 2011) and item 7 (administered in 2007, 2011, and 2015) both measure misconception P3A (“gravity acts only on falling objects, but not on objects at rest or on objects that are moving upward”). For MC item 4 (“gravity acting on a parachute jumper”), the percentage of students demonstrating the misconception was not measurably different in 2011 than in the previous two assessments in Italy, Norway, the Russian Federation, and the United States. In contrast, in Slovenia, the percentage of students decreased between 2003 (55%) and 2011 (46%). For CR item 7 (“forces acting on people sitting on a wall”), there were significant differences in Italy and the United States but not in the other countries. In both Italy and the United States, the percentage of students with the misconception in 2011, the second assessment cycle, was higher than in 2015, while the percentage in 2007 was not measurably different. This reflected an 8% decrease in the percentage of students in Italy and a 5% decrease in the United States between 2011 and 2015. The largest differences were found for item 9 (“force causing a ball thrown upward to fall”), which measured misconception P3B (“gravity alone cannot cause an object initially at rest to start moving; it requires another force/push”). In both the Russian Federation and Slovenia, the percentage of students demonstrating the misconception decreased over time, and the difference between the first assessment cycle (2007) and the third (2015) was statistically significant. In Slovenia, this reflected a 10% decrease (from 37 to 27%) and in the Russian Federation, a 12% decrease (from 38 to 26%), with no measurable differences in Italy, Norway, and the United States.

At grade four (Fig. 4.24), the trend item data also covered assessment years 2003, 2007, 2011, and 2015. There were fewer significant differences over time in the percentage of students with misconceptions on the grade four items than at grade eight. As at grade eight, however, the most substantial differences were found in the Russian Federation and Slovenia. Items 10 and 11 both measure misconception P3B (“gravity alone cannot cause an object initially at rest to start moving; it requires another force/push”). On MC item 10 (“force causing object to fall to the ground”), the only countries with a significant difference were the Russian Federation and the United States. In the Russian Federation, the percentage of students demonstrating the misconception was 7% lower in 2011 (19%) than in 2007 (26%). In the United States, the percentage was lower in 2011 (7%) than in 2003 (11%), although the decrease appears to have occurred between 2003 and 2007 (also 7%). On CR item 11 (“force causing a marble to roll down a sloping track”), the percentage of students demonstrating the misconception decreased significantly in the Russian Federation (from 67 to 47%) and Slovenia (from 90 to 84%) between the 2011 and 2015 assessments. Item 13 (“direction of the force of Earth’s gravity”) measures misconception P4B (“gravity pushes upward on objects sitting on a solid surface and on objects that are moving upward”). There was only one statistically significant difference on this item, and that was a slight decrease of 2% of students in the Russian Federation between the 2011 and 2015 assessments (from 5% to 3%).

### 4.2.6 Summary of Physics Results

In the physics results sections, we have reported on students’ performance on the set of items related to gravity across countries at each grade level (TIMSS Advanced, grade eight, and grade four; see Sect. 4.2.1), patterns in student misconceptions and misunderstandings across countries and grade levels (Sects. 4.2.2 and 4.2.3), gender differences in these misconceptions and misunderstandings (Sect. 4.2.4), and trends over multiple assessment years (Sect. 4.2.5). The frequency of specific types of student misconceptions and misunderstandings at each grade level varied across the five countries included in the study. In each country, and at each grade level, there were some misconceptions and misunderstandings that were demonstrated by at least one-third of students. Gender differences were found at all three grade levels, but were most prevalent on the TIMSS Advanced items. For all trend items (except one in TIMSS Advanced), gender differences favored males, with higher percentages of female students than male students demonstrating the misconception or misunderstanding. Performance on grade eight and grade four trend items administered in multiple assessment years showed that the frequency of certain student misconceptions and misunderstandings decreased over time in some countries but not others.

## 4.3 Mathematics Results

^{19}19 TIMSS items at grade eight, and eight TIMSS items at grade four. We identified nine performance objectives (POs) related to linear equations that were measured by these items, each with a specific set of related errors and misunderstandings (Table 4.21).

Mathematics performance objectives related to linear equations with related errors and misunderstandings, by grade level

Performance objective | Related errors and/or misunderstandings | Grade level | ||
---|---|---|---|---|

TA | Gr8 | Gr4 | ||

MO1: Interpret the solution to a system of linear equations to answer a question or solve a problem in real life | Not able to use slope and intercept to provide an argument in support of the solution to a real-life problem situation (M1) | ✓ | ||

MO2: Solve systems of linear equations in two variables | Not able to apply the procedure correctly to solve a real-life problem situation (M2A) | ✓ | ✓ | |

Not able to apply the procedure correctly to solve non-contextualized problems (M2B) | ✓ | |||

MO3: Interpret the meanings of slope and y-intercept in linear equations or graphs | Not able to relate slope with steepness of a line. (M3A) | ✓ | ||

Demonstrates confusion between slope and intercept of an equation. (M3B) | ✓ | |||

MO4: Relate algebraic equations to their graphical representations (and vice versa) | Not able to correctly identify the graph of an equation (M4A) | ✓ | ||

Not able to translate graphical representations into a mathematical equation or verbal description of a linear relationship (M4B) | ✓ | |||

MO5: Write equations to represent situations | Not able to translate verbal descriptions into a correct mathematical equation (M5) | ✓ | ||

MO6. Given pairs of numbers in tables or ordered pairs, generate an algebraic equation of the relationship between two variables | Not able to translate relationship shown in table form into a mathematical equation (M6) | ✓ | ||

MO7 Given pairs of numbers in tables or ordered pairs, generate a verbal description of the relationship | Not able to generate a correct verbal description given a specific relationship in the form of ordered pairs (M7A) | ✓ | ||

Not able to generate a correct verbal description given a specific relationship shown in table form (M7B) | ✓ | |||

MO8: Give a verbal description of a relationship between a set of numbers, generate pairs of whole numbers that follow that relationship (rule) | Not able to identify a correct set of numbers that follow a given relationship/rule (M8) | ✓ | ||

MO9: Apply algebraic thinking to solve simple real-life problems involving unknowns | Not able to apply algebraic thinking to solve simple real-life problems involving unknowns (M9) | ✓ |

List of TIMSS and TIMSS Advanced mathematics items related to linear equations, organized by performance objective and grade level

Item | Grade level | Assessment year(s) | Item format | Item description | Figure | Mathematics errors and misunderstandings | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

M1 | M2A | M2B | M3A | M3B | M4A | M4B | M5 | M6 | M7A | M7B | M8 | M9 | ||||||

| ||||||||||||||||||

Item 1B (MA33240B) | TIMSS Advanced | 2015 | CR | Compare car rental plans X and Y—interpret solution | 4.28 | ✓ | ||||||||||||

| ||||||||||||||||||

Item 1A (MA33240A) | TIMSS Advanced | 2015 | CR | Compare car rental plans X and Y—solve system of equations | 4.28 | ✓ | ||||||||||||

Item 2 (M042263) | Grade 8 | 2007 | CR | Cost of 1 pen and 2 pencils | 4.29 | ✓ | ||||||||||||

Item 3 (M062237) | 2015 | CR | Solve system of equations for x and y | 4.30 | ✓ | |||||||||||||

Item 4 (M052087) | 2011, 2015 | CR | Solve for simultaneous equations | – | ✓ | |||||||||||||

Item 5 (M032728) | 2003 | MC | If x plus 3y equals 11 and 2× plus 3y equals 13 | – | ✓ | |||||||||||||

| ||||||||||||||||||

Item 6 (M052105) | Grade 8 | 2011, 2015 | CR | Which slope has a greater value | – | ✓ | ||||||||||||

Item 7 (M012022) | 1995, 1999 | MC | Equation to determine cost of cards | – | ✓ | |||||||||||||

| ||||||||||||||||||

Item 8 (M062095) | Grade 8 | 2015 | MC | Equation for Bruno’s monthly cell phone cost | – | ✓ | ||||||||||||

Item 9 (M062074) | 2015 | MC | Function for Janet’s graph given slope/intercept | 4.31 | ✓ | |||||||||||||

| ||||||||||||||||||

Item 10 (M052092) | Grade 8 | 2011, 2015 | MC | Graph of y = 2x | – | ✓ | ||||||||||||

Item 11 (M042112) | 2007, 2011, 2015 | MC | Relationship between x and y | 4.32 | ✓ | |||||||||||||

Item 12 (M062242) | 2015 | MC | Relationship a graph of a line in words | – | ✓ | |||||||||||||

| ||||||||||||||||||

Item 13 (M042202) | Grade 8 | 2007, 2011, 2015 | MC | Formula for K the cost of trip | 4.33 | ✓ | ||||||||||||

Item 14 (M062241) | 2015 | CR | Roy’s phone business—equation for y | – | ✓ | |||||||||||||

Item 15 (M032545) | 2003 | CR | 7 oranges and 4 lemons cost 43 zeds | – | ✓ | |||||||||||||

Item 16 (M052090) | 2011, 2015 | MC | Set up system of equations | – | ✓ | |||||||||||||

| ||||||||||||||||||

Item 17 (M032163) | Grade 8 | 2003, 2007 | MC | Relation between x and y in a table | 4.34 | ✓ | ||||||||||||

Item 18 (M022042) | 1999 | MC | Correct equation based on x/y table | – | ✓ | |||||||||||||

Item 19 (M012046) | 1995, 1999 | MC | Equation from x/y table | 4.35 | ✓ | |||||||||||||

| ||||||||||||||||||

Item 20 (M012029) | Grade 8 | 1995, 1999, 2003 | MC | Sets of ordered pairs of numbers | 4.36 | ✓ | ||||||||||||

Item 21 (M041125) | Grade 4 | 2007, 2011, 2015 | MC | Rule to get numbers in column B | 4.37 | ✓ | ||||||||||||

Item 22 (M031227) | 2003, 2007 | CR | Sean’s rule to transform numbers | 4.38 | ✓ | |||||||||||||

Item 23 (M031251) | 2003, 2007, 2011 | MC | Ebru’s rule to get number in box | – | ✓ | |||||||||||||

| ||||||||||||||||||

Item 24 (M041124) | Grade 4 | 2007, 2011, 2015 | CR | Use the rule to complete the table | 4.39 | ✓ | ||||||||||||

Item 25 (M061254) | 2015 | CR | Complete the table for Mia’s input-output game | – | ✓ | |||||||||||||

Item 26 (M031242A) | 2003, 2007 | CR | Renting bikes/fill table | 4.40 | ✓ | |||||||||||||

| ||||||||||||||||||

Item 27 (M031247) | Grade 4 | 2003, 2007 | CR | A man took his children to a fair | 4.41 | ✓ | ||||||||||||

Item 28 (M051006) | 2011, 2015 | CR | Cost of ice cream | 4.42 | ✓ |

### 4.3.1 Student Performance on TIMSS and TIMSS Advanced Items Related to Linear Equations

^{20}Average performance across the five countries for the TIMSS Advanced items (Fig. 4.25) ranged from 37 to 43% correct. In comparison, the average item performance on grade eight items (Fig. 4.26) ranged from 13 to 63% correct and on grade four items (Fig. 4.27) ranged from 17 to 62% correct. Some notable differences in performance were observed across the five countries.

For TIMSS Advanced (Fig. 4.25), the widest-ranging item performance was in Norway (from 43 to 62% correct) and the United States (from 40 to 57% correct). In contrast, item-level performance ranged from 36 to 40% correct in the Russian Federation, from 22 to 25% correct in Italy, and from 36 to 39% correct in Slovenia. In three of the five countries, students found item 1B more difficult than item 1A. In contrast, in Italy and Slovenia, 3% more students found item 1A more difficult than item 1B.

At grade eight (Fig. 4.26), a broad range of item performance was found in all five countries, with the lowest range (43 percentage points) in Norway and the highest range (61 percentage points) in the Russian Federation and the United States. The three most difficult items in all countries were CR items. Two of them, items 2 and 3, are from performance objective 2 (“solve system of linear equations in two variables”) and one, item 6, is from performance objective 3 (“interpret the meanings of the slope and y-intercept in linear equations and graphs”). For both items 2 and 3, item performance was lowest in Norway (4% and 1% correct, respectively) and highest in the Russian Federation (36% correct on both items). On item 6, performance was lowest in Italy (2% correct) and highest in the United States (31% correct). In comparison, the two easiest items (item 12 and item 13) were both MC items. Item 12 is from performance objective 4 (“relate algebraic equations to their graphical representations (and vice versa)”) and item 13 is from objective 5 (“write equations to represent situations”). For items 12 and 13, performance was lowest in Norway (44% and 41% correct, respectively) and highest in the United States (78% and 75% correct, respectively).

There was also a broad range of item performance found in each country at grade four (Fig. 4.27). The smallest range in percent correct across grade four items was in the United States (23–66%) and the largest was in the Russian Federation (22–81%). Interestingly, at grade four, the three most difficult items across countries (items 22, 27, and 28) as well as the two of the easiest items (items 23 and 26) were CR items. In comparison, at grade eight, the easiest items were all MC items. One of the most difficult items at grade four, item 22, is from performance objective 7 (“given pairs of numbers in tables or ordered pairs, generate a verbal description of the relationship”). The performance on this item varied from 8% correct in Slovenia to 23% correct in the Russian Federation and the United States. The other two items are from performance objective 9 (“apply algebraic thinking to solve simple real-life problems involving unknowns”). The items that were easy in general for all countries were from performance objective 8 (“given a verbal description of a relationship between a set of numbers, generate pairs of whole numbers that follow that relationship/rule”).

### 4.3.2 Common Types of Errors and Misunderstandings Related to Linear Equations Across Countries

*y*-intercepts remains the same. Therefore, the distance on the

*x*-axis at which the two equations intersect will not change. A common misunderstanding is that the students do not understand the solution well enough to go beyond the set procedure and explain the solution with a change in the initial cost of the rental plan.

Student performance data for mathematics item 1 (MA33240A and MA33240B), 2015

Students received partial credit for providing a correct response to either part A (code 10) or part B (code 11). An incorrect response to part A (code 11, 79, or 99) demonstrates that students cannot evaluate the context, write equations, or apply the correct procedures to solve a system of equations. On average, 58% of students demonstrated this error across the five countries. The percentage of students making this error varied considerably across the five countries, from 79% of students in Italy, to 38% of students in Norway and 43% of students in the United States.

An incorrect response to part B (code 10, 79, or 99) demonstrates that students did not understand the system of equations well enough to explain the impact of the *y*-intercept change on both equations. That is, they did not demonstrate a deeper understanding beyond applying procedures to solve a pair of linear equations. On average, 64% of students demonstrated this error across the five countries. Italy had the highest percentage of students (76%) demonstrating this misunderstanding, and Norway had the lowest percentage (57%).

Student performance data for mathematics item 2 (M042263), 2007

Student performance data for mathematics item 3 (M062237), 2015

Another performance objective is that, by the end of grade eight, students should be able to identify and interpret the slope and the intercepts in linear equations shown algebraically and graphically. Item 6 (not shown as it is a secured item) includes two lines on a graph and their equations. Students were expected to determine which line had a greater slope by relating the steepness of the line with the slope of the linear equation. On average across the five countries, nearly 86% of the students failed to correctly identify which slope was larger. In all countries except the United States (69%), more than 85% of the students were not able to correctly relate that the steeper line has the larger slope.

Student performance data for mathematics item 9 (M062074), 2015

Another performance objective is that by the end of grade eight, students should be able to translate between algebraic and graphical representations. In item 10 (not shown as it is a secured item), students need to identify the graph of a given equation. On average, across the five countries, only 23% of students correctly identified the graph of the equation given in the item; students in Norway were least likely to get this correct (10%), while the highest percentage of students getting this correct was in the Russian Federation (48%).

Student performance data for mathematics item 11 (M042112), 2015

Student performance data for mathematics item 13 (M042202), 2015

^{21}also required students to identify the correct equation of a line from the relationship given in a table. However, in this case, two response options (options B and C) are not equations for a single straight line. Hence, the choice is between the other two options.

Student performance data for mathematics item 17 (M032163), 2007

Student performance data for mathematics item 19 (M012046), 1999

Student performance data for mathematics item 20 (M012029), 2003

Student performance data for mathematics item 21 (M041125), 2015

Student performance data for mathematics item 22 (M031227), 2007

Student performance data for mathematics item 24 (M041124), 2015

Student performance data for mathematics item 26 (M031242A), 2007

Student performance data for mathematics item 27 (M031247), 2007

Student performance data for mathematics item 28 (M051006), 2015

### 4.3.3 Patterns in Errors and Misunderstandings Related to Linear Equations Across Grade Levels and Countries

^{22}

The first performance objective is to interpret the solution to a system of linear equations to answer a question or solve a problem in a real-life context. On average across the five countries, 63% of TIMSS Advanced students (Fig. 4.43) demonstrated misunderstanding M1 (“not able to use slope and intercept to provide an argument in support of the solution to a real-life problem situation”) on item 1B (Fig. 4.28). In the Russian Federation, Slovenia, and the United States, the percentage of students with this misunderstanding was similar to the average. In Italy, the percentage of students demonstrating this misunderstanding was higher (75%) and, in Norway, it was lower (57%) than the average.

Under performance objective 2 (“solve systems of linear equations in two variables”), there are two types of errors, depending on whether students are applying the procedure to a contextualized real-life problem (M2A) or to a non-contextualized problem (M2B). In a contextual situation, students need an additional piece of understanding to evaluate the situation and write the correct equation. This is not needed in the case of a non-contextual situation. There are two example items involving contextualized problems, one from TIMSS Advanced (Fig. 4.28, item 1A) and one from grade eight (Fig. 4.29, item 2). Many students at both grade levels were not able to apply the procedure correctly (error M2A). On average across the five countries, 57% of TIMSS Advanced students and 72% of grade eight students made this error (Figs. 4.43 and 4.44). In Norway, the Russian Federation, and the United States, the percentage of students making error in the procedure was lower in TIMSS Advanced than in grade eight (as might be expected), but in Italy and Slovenia, the percentage of students was comparable at both grade levels. There was more variation across countries in the percentage of students making this error among TIMSS Advanced students (a range of 40 percentage points) than among grade eight students (a range of 17 percentage points). In TIMSS Advanced, the percentage of upper-secondary students not able to apply the procedure ranged from 38% of students in Norway and 43% of students in the United States, to 60% in the Russian Federation and 64% in Slovenia, to 78% in Italy.

The related error (M2B) occurred when students were not able to apply the procedure correctly to solve a non-contextualized problem. There were three items at grade eight, item 3 (Fig. 4.30), item 4 (not shown), and item 5 (not shown), that assessed whether students could correctly apply the procedure to solve a system of linear equations (Fig. 4.44). The average percentage of students making errors in applying the procedure was very high across countries on all three items (84%, 86%, and 62% on items 3, 4, and 5, respectively). As expected, this was more pronounced in the CR items (items 3 and 4) than in the MC item (item 5). The pattern across items was very similar in all counties except the Russian Federation. In the case of the Russian Federation, the difference between CR and MC items was less pronounced (64%, 60%, and 56% on items 3, 4, and 5, respectively).

Another important and prevalent misunderstanding among grade eight students is not being able to relate the steepness of a line with the slope of the line (M3A). The percentage of students demonstrating this misunderstanding on the CR item 6 (not shown) was very high in all countries, with four of the five countries having at least 86% of students in this category. In comparison, 69% of students in the United States demonstrated this misunderstanding (Fig. 4.44).

A related misunderstanding that students demonstrate is confusion between the slope and intercept of an equation (M3B). At grade eight, there are three MC items included in this set that are related to this misunderstanding: item 7 (not shown), item 8 (not shown), and item 9 (Fig. 4.31). For all three items, this specific misunderstanding was tracked by having a distractor in which the two values for intercept and slope were swapped in the equations. This misunderstanding was found to be lower among grade eight students than were some of the other misunderstandings and/or errors (ranging from 10 to 24%, on average). This misunderstanding was less common among students in the Russian Federation (demonstrated by 7–18% of students, depending on the item), but more common in Italy (11–28%), Norway (29–31%), Slovenia (20–24%), and the United States (13–22%).

Performance objective 4 is related to students being able to translate easily between an algebraic equation and the graph of a line. The first misunderstanding related to this performance objective is that students are not able to correctly identify the graph of an equation (M4A). More than 75% of students in Italy, Norway, and Slovenia demonstrated this misunderstanding and were not able to identify the correct graph for a given equation on item 10 (not shown).

A related error is not being able to write or identify the correct algebraic equation or verbal description from the graph of a line (M4B). Students found translating the graph of a line to its algebraic form (item 11: Fig. 4.32) more difficult than translating the graph of a line to a verbal description of the relationship on item 12 (not shown). On average across the five countries, 55% of students were not able to select the correct equation for the given graph of a line (item 11), and 37% of students were not able to select the correct description of the relationship/rule for the given graph of a line (item 12). The percentage of students demonstrating the error (M4B) on item 11 varied from 70% in Norway to 45% in Slovenia. In contrast, students demonstrating the error (M4B) on item 12 varied from 56%, again in Norway, to 22% in the United States. This means students understood the relationship between the two variables but found it difficult to verbalize the relationship in algebraic form. The difference between the error demonstrated on items 11 and 12 was most pronounced in the Russian Federation (20 percentage points) and the United States (37 percentage points) and least pronounced in Italy (5 percentage points).

The next type of misunderstanding (M5) was not being able to translate verbal descriptions into a correct mathematical equation. There were four grade eight items measuring this error: two MC (item 13 in Fig. 4.33 and item 16, not shown) and two CR (items 14 and 15, not shown). As was found with other errors and misunderstandings, the percentage of students demonstrating the misunderstanding was higher on the CR items than on the MC items in all five countries. On all four items, the percentage of students demonstrating the misunderstanding was highest in Norway and lowest in the Russian Federation (except for item 13). On item 13, the percentage of students demonstrating the misunderstanding was lowest in United States.

The related misunderstanding (M6) was not being able to translate the relationship given in a table format into a linear equation. All three grade eight items assessing this misunderstanding were MC in format: item 17 (Fig. 4.34), item 18 (now shown), and item 19 (Fig. 4.35).^{23} Across all three items, the misunderstanding was more common among students in Italy than among students in the Russian Federation and the United States. Another related misunderstanding (M7A) is not being able to generate a verbal description given a specific relationship in the form of ordered pairs, which was measured on one grade eight item. The percentage of grade eight students demonstrating this error on item 20 was highest in Norway (60%) and lowest in the Russian Federation and the United States (31% and 32%, respectively).

The next three kinds of errors and misunderstandings were demonstrated at grade four (Fig. 4.45). The first one (M7B) is not being able to generate a verbal description given a relationship in table format. There are three items included in this set (items 21, 22, and 23). Item 21 (Fig. 4.37) and item 23 (not shown) are MC format, and item 22 (Fig. 4.38) is CR format. The misunderstanding appears to be more prevalent on the CR item than on the MC items for all countries. The misunderstanding was more common in Norway (68–91% of students across items) and Slovenia (60–92%) than in Italy (47–78%), the Russian Federation (37–77%), and the United States (36–77%).

Another related error (M8) was not being able to identify a correct set of numbers based on the verbal description of the relationship. There were three grade four CR items that measured this misunderstanding: item 24 (Fig. 4.39), item 25 (not shown), and item 26 (Fig. 4.40). The percentage of students demonstrating this type of error on item 24 covered a range of 34 percentage points across the five countries, from 19% in the Russian Federation to 53% in Norway. Similarly, the percentage of students demonstrating this type of error for item 25 covered a range of 30 percentage points, from 28% in the Russian Federation to 58% in Norway. For item 26, the percentage of students demonstrating this error was more consistent across countries, ranging from 29% in Italy to 35% in Norway.

The last type of error or misunderstanding (M9) was that students were not able to apply algebraic thinking to solve simple real-life problems, which is a precursor skill for linear equations. Both example items 27 (Fig. 4.41) and 28 (Fig. 4.42) were CR in format. In responding to these items, students were not expected to formally write equations but to apply algebraic thinking to solve them. Two-thirds of students on average across the five countries (and at least half in each country) were not able to solve these problems correctly.

### 4.3.4 Gender Differences in Errors and Misunderstandings Related to Linear Equations

^{24}and the percentage of students with specific errors and misunderstanding by gender (Tables 4.39, 4.40, and 4.41, and Figs. 4.46, 4.47, and 4.48)

^{25}differed across countries and grade levels. Of the few gender differences observed for linear equation items, more favored males than females.

Performance of female and male students on TIMSS and TIMSS Advanced mathematics items, by country and grade level, 1999–2015

Percentage of female and male TIMSS Advanced students with errors and misunderstandings about linear equations, by country, 2015

Percentage of female and male grade eight students with errors and misunderstandings about linear equations, by country: 1999, 2003, 2007, and 2015

Percentage of female and male grade four students with errors and misunderstandings about linear equations, by country, 2007, 2011, and 2015

Gender differences in percent correct were greatest on the TIMSS Advanced items, with an average female-male difference of at least 8% in favor of boys on both items (Table 4.38). However, this varied across countries, with significant differences found in Italy, the Russian Federation, and Slovenia, but not in Norway or the United States. In comparison, on average across the 19 items at grade eight, significant female-male differences in item percent correct ranged from 4–7%, with two items having measurably different performance that favored boys and two items that favored girls. In all countries, there were two or three grade eight items with significant gender differences. On average at grade four, only one of the eight items had a significant item performance difference that favored males (by 4%). However, in Norway, there were three items where the percent correct favored males.

The specific set of items with significant gender differences varied across countries. None of the items had significant gender differences in all five countries, but the two TIMSS Advanced items had significant gender differences in three countries. In contrast, 12 of the 19 items at grade eight and five of the eight items at grade four had significant gender differences in one or two countries.

Looking at the percent correct in each country, Italy had significant gender differences that favored males on both items in TIMSS Advanced and one item each at grade eight (item 17) and grade four (item 28); grade eight females in Italy performed better on one item (item 7). Norway had one item at grade eight (item 12) and three items at grade four (items 23, 26, and 28) where males performed better than females, and one item at grade eight (item 8) where females performed better than males. The Russian Federation had significant gender differences in favor of males on both items in TIMSS Advanced. In grade eight, there were two items (items 6 and 12) where males performed better than females and two items (items 15 and 19) where females performed better than males. There were no significant gender differences on any grade four items in the Russian Federation. Slovenia had both items in TIMSS Advanced, two items at grade eight (items 3 and 10), and one item at grade four (item 27) where males performed better than females, and one item at grade eight (item 20) where females performed better than males. In comparison, the United States was the only country with no items where male performance was higher; there were two items at grade eight (items 7 and 13) and one item at grade four (item 24) where females did better than males.

The United States was the only country with significant gender differences on items 8 and 13 (misunderstanding/error M3B and M5), with both items having a higher percentage of males with the misunderstanding/error (5–6%). In contrast, the Russian Federation exhibited gender differences on five of the 19 items in the set. Two items (6 and 12) showed a higher percentage of females in the Russian Federation demonstrating the misunderstanding/error (M3A and M4B), and three items (7, 15, and 19) showed a higher percentage of males demonstrating the misconceptions assessed by these items (M3B, M5, and M6).

Italy exhibited gender differences for two of the 19 items. On item 7, 5% more males than females had the misunderstanding (M3B), while on item 17, 12% more females had the misunderstanding (M6). In Norway, only item 12 had a significant difference, with 10% more females than males demonstrating the error (M4B).

### 4.3.5 Patterns in Errors and Misunderstandings Related to Linear Equations Over Time

Looking at grade eight (Fig. 4.49), the trend item data covered assessment years 1995, 1999, 2003, 2007, 2011, and 2015. There are some significant differences across assessment years in the percentage of students in each country demonstrating the specific types of errors or misunderstandings. Item 4 (administered in 2011 and 2015) measures error M2B (“not able to apply the procedure correctly to solve non-contextualized problems”). This item shows a decrease of 4% of students making the error in the United States in 2015 in comparison to the previous assessment cycle in 2011.

Item 6 (administered in 2011 and 2015) measures misunderstanding M3A (“not able to relate slope with steepness of lines”), and the trend data for this item show an increase in the percentage of students demonstrating this misunderstanding over time for students in Italy (4%) and the Russian Federation (7%), but a decrease for the United States (9%), Norway (5%), and Slovenia (8%). Item 7 (administered in 1995 and 1999) measures the related misunderstanding M3B (“demonstrates confusion between slope and intercept of an equation”). Trend data are available for only two countries, the Russian Federation and the United States. The percentage of students demonstrating this misunderstanding did not change between 1995 and 1999 for either country.

Item 10 (administered in 2011 and 2015) measures error M4A (“not able to correctly identify the graph of an equation”). The percentage of students making this error decreased by 7% in the United States. Another related error M4B (“not able to translate graphical representations into a mathematical equation or verbal description of a linear relationship”) was measured by item 11 (administered in 2007, 2011, and 2015). The trend data for this item show no change in the percentage of students demonstrating the error from 2011 for all countries. However, for Norway and the United States, the percentage of students demonstrating the misconception increased after the 2007 cycle of TIMSS.

Item 13 (administered in 2007, 2011, and 2015) and item 16 (administered in 2011 and 2015) both measure error M5 (“not able to translate verbal descriptions into a correct mathematical equation”). For item 13 (“formula for K the cost of trip”), the general trend was a decrease in the percentage of students demonstrating this error in 2015 from the previous two assessment years (based on the average percentage across the five countries). However, in Italy and Norway, the percentage appeared to increase between 2007 and 2011 and then decrease between 2011 and 2015 (though the differences were not statistically significant). For item 16 (“set up system of equations”), the difference in the percentage of students did not change between 2011 and 2015.

Item 17 (administered in 2003 and 2007) and item 19 (administered in 1995 and 1999) both measure misunderstanding M6 (“not able to translate relationship shown in table form into a mathematical equation”). Trend data for item 17 show no statistically significant difference in the percentage of students demonstrating this misunderstanding from 2003 to 2007. For item 19, trend data were only available for the Russian Federation and the United States. On this item, the percentage of students demonstrating the misunderstanding decreased between 1995 and 1999 by 10% in the United States and was not statistically different in the Russian Federation.

Item 20 (administered in 1995, 1999, and 2003) measures misunderstanding M7A (“not able to generate a correct verbal description given a specific relationship in the form of ordered pairs”). For this item, complete data for three assessment cycles were only available for the Russian Federation and the United States. In contrast, data for Italy are available for 1999 and 2003 and data for Norway and Slovenia are available for 1995 and 2003. The available data show that the percentage of students with this misunderstanding increased over time in Norway (by 12%) and Slovenia (8%), but decreased in the United States (by 5%).

At grade four, the trend item data covered assessment years 2003, 2007, 2011, and 2015 (Fig. 4.50). Again, at grade four, there were some significant differences over time in the percentage of students demonstrating the misunderstandings. Item 21 (administered in 2007, 2011, and 2015), item 22 (administered in 2003 and 2007), and item 23 (administered in 2003, 2007, and 2011) all measure misunderstanding M7B (“not able to generate a correct verbal description given a specific relationship shown in table form”). In general, across countries the data show a decrease in students having this misunderstanding. Trend data for item 21 show a significant decrease in the percentage of Slovenian students showing the misunderstanding from 2007 to 2015 (by 7%). On item 22, the percentage of students having this misunderstanding decreased from 2003 to 2007 in the Russian Federation (by 15%) and the United States (by 7%). Similarly, in the case of item 23, the percentage of students having this misunderstanding decreased significantly between 2003 and 2011, by 8% in Italy, and by 11% in the Russian Federation and the United States.

Item 24 (administered in 2007, 2011, and 2015) and item 26 (administered in 2003 and 2007) measure misunderstanding M8 (“not able to identify a correct set of numbers that follow a given relationship/rule”). Trend data for item 24 show some interesting patterns over these three points in time. The percentage of students showing this misunderstanding decreased consistently over time in Norway (by 12%), the Russian Federation (16%), and Slovenia (7%). Trend data for item 26 show that the percentage of students with this misconception generally decreased from 2003 to 2007, with significant decreases in Slovenia (by 13%) and the Russian Federation (14%).

Item 27 (administered in 2003 and 2007) and item 28 (administered in 2011 and 2015) measure misunderstanding M9 (“not able to apply algebraic thinking to solve simple real-life problems involving unknowns”). Trend data for item 27 showed that the percentage of students demonstrating the misunderstanding decreased from 2003 to 2007 in the Russian Federation (by 11%) and was not significantly different over time in the other countries. Data for item 28 showed that the percentage of students with this misconception decreased from 2011 to 2015 only for Slovenia (by 6%).

### 4.3.6 Summary of Mathematics Results

We have reported students’ performance on the set of items related to linear equations across countries at each grade level (TIMSS Advanced, grade eight, and grade four; Sect. 4.3.1), patterns in student errors and misunderstandings across countries and grade levels (Sects. 4.3.2 and 4.3.3), gender differences in these errors and misunderstandings (Sect. 4.3.4), and trends over multiple assessment years (Sect. 4.3.5). The frequency of specific types of student errors and misunderstandings at each grade level varied across the five countries included in the study. In each country, and at each grade level, there were some errors and misunderstandings that were demonstrated by at least 50% of the students. There were some gender differences at all three grade levels. Most of the measurable gender differences favored males (i.e., a smaller percentage of males than females demonstrated the error or misunderstanding measured by the item), but there were some that favored females (primarily at grade eight). Performance on trend items administered in multiple assessment years showed that the frequency of certain student errors and misunderstandings changed over time. Performance on grade eight items showed a decrease for some errors and misunderstandings and an increase for others in some countries. Some measurable decreases were also observed at grade four, but, in contrast to grade eight, there were no items that showed an increase in the percentage of students demonstrating the error or misunderstanding.

## Footnotes

- 1.
See Sect. 1.2 for definitions of the terminology used throughout the report (performance objectives, misconceptions, errors, and misunderstandings) and how these relate to the physics and mathematics items.

- 2.
Each country translates the international version of the TIMSS and TIMSS Advanced assessment items into their language(s) of instruction, and these translated national versions are verified by the IEA.

- 3.
As noted previously, results are based on both released and non-released items that measure the specific types of misconceptions, misunderstandings, and errors. Example items presented in the report are all restricted-use items (from 2015) and released items (from previous assessments). For trend items administered in multiple assessments, the data shown in the tables and figures in this section reflect the most recent assessment year (shown in Tables 4.2 and 4.22).

- 4.
Gender differences on TIMSS Advanced items are based on the sample of students who took the TIMSS Advanced assessment. As described in Chap. 3, the TIMSS Advanced population reflects a select group of students in each country, and the percentage of female and male students taking more advanced courses may differ from the percentages in the full population of students in their final year of secondary school (see Tables 3.4 and 3.5).

- 5.
Trend graphs are included only for grades four and eight, as there were no TIMSS Advanced trend items available that measured the specific physics and mathematics concepts under study.

- 6.
- 7.
Item 2 is a released item from the 1995 assessment. Comparable data are not available from Italy. Thus, this item is only included for the other four countries (Norway, the Russian Federation, Slovenia, and the United States).

- 8.
Item 2 is a released TIMSS Advanced item from 1995. Data for Italy are not available; thus, the international average is based on the other four countries (Norway, the Russian Federation, Slovenia, and the United States).

- 9.
Item 3 is a released TIMSS Advanced item from 2008. The United States did not participate in the 2008 assessment; thus, the international average is based on the other four countries (Italy, Norway, the Russian Federation, and Slovenia).

- 10.
This is based on the unrounded data (not shown in Table 4.6). Data to the nearest 0.01% are available at www.iea.nl/publications/RfEVol9.

- 11.
These two items are not shown as exhibits in the report, but both are released items available from the IEA website (see www.iea.nl).

- 12.
Item 5 is a released TIMSS item from 1999. Norway did not participate in the 1999 assessment, and comparable data are not available for Slovenia. Thus, the international average is based on the other three countries (Italy, the Russian Federation, and the United States).

- 13.
Item 6 is a released TIMSS item from the 1995 assessment. Comparable data are not available for Italy. Thus, the international average is based on the other four countries (Norway, the Russian Federation, Slovenia, and the United States).

- 14.
Item 14 is a secure item from TIMSS 2015 and cannot be shown in the report.

- 15.
Item 14 is a secure item from TIMSS 2015 and cannot be shown in the report.

- 16.
- 17.
- 18.
Tables 4.18, 4.19 and 4.20 display the percentage of female and male students with each misconception in TIMSS Advanced, grade eight and grade four, respectively. The accompanying figures (Figs. 4.20, 4.21, and 4.22) illustrate the differences in the percentages of female and male students at the corresponding grade levels. The corresponding overall percentages of students with the misconceptions are shown in Figs. 4.17, 4.18, and 4.19.

- 19.
Parts A and B of TIMSS Advanced item 1 are treated as separate items in this report (items 1A and 1B).

- 20.
- 21.
Slovenia and Norway did not participate in TIMSS 1999 assessment. Hence, average performance for this item is based on data from three countries (Italy, the Russian Federation, and the United States).

- 22.
- 23.
Norway and Slovenia did not participate in the 1999 and 1995 assessments.

- 24.
- 25.
Tables 4.39, 4.40 and 4.41 display the percentage of female and male students with each error or misunderstanding in TIMSS Advanced, grade eight, and grade four, respectively. The accompanying figures (Figs. 4.46, 4.47, and 4.48) provide graphical displays of the differences in the percentage of female and male students at the corresponding grade level. The corresponding percentages of students overall with the errors or misunderstandings are shown in Figs. 4.43, 4.44, and 4.45.

## References

- Sneider, C., & Pulos, S. (1983). Children’s cosmographies: Understanding the Earth’s shape and gravity.
*Science Education,**67*(2), 205–221.CrossRefGoogle Scholar

## Copyright information

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.