Results for Student Misconceptions, Errors, and Misunderstandings in Physics and Mathematics

Neidorf, Teresa; Arora, Alka; Erberber, Ebru; Tsokodayi, Yemurai; Mai, Thanh

doi:10.1007/978-3-030-30188-0_4

Teresa Neidorf²³,
Alka Arora²³,
Ebru Erberber²³,
Yemurai Tsokodayi²³ &
…
Thanh Mai²³

Part of the book series: IEA Research for Education ((IEAR,volume 9))

12k Accesses
1 Citations

Abstract

Diagnostic item-level student performance data from twenty years of TIMSS and TIMSS Advanced assessments can be used to explore students’ level of understanding of gravity and linear equations across grades four, eight, and the final year of secondary school (TIMSS Advanced students). Sets of assessment items at each grade level illustrate the nature and extent of student misconceptions, errors, and misunderstandings across grade levels in five countries (Italy, Norway, the Russian Federation, Slovenia, and the United States). The results include assessment of how students in each country performed on the set of items measuring understanding of the physics and mathematics concepts explored in this study (gravity and linear equations); common types of student misconceptions, errors, and misconceptions across grade levels in each country; patterns in misconceptions, errors, and misunderstandings across countries; and gender differences in the frequency of misconceptions, errors, and misunderstandings in each country. The frequency of specific types of student misconceptions, errors, and misunderstandings at each grade level varied across the five countries. Gender differences were found at all three grade levels, but were more extensive for physics than mathematics. Trend items administered in multiple assessment years indicated that the frequency of certain student misconceptions, errors, and misunderstandings decreased over time, while the frequency of others increased.

You have full access to this open access chapter, Download chapter PDF

Keywords

4.1 Introduction

The results for physics (Sect. 4.2) and mathematics (Sect. 4.3) start with an overview of the set of TIMSS and TIMSS Advanced items that measure student understanding of the key concepts that are the focus of this study (gravity in physics and linear equations in mathematics). The overview also describes the performance objectives that are assessed by the items across grade levels and the types of student misconceptions, errors, and misunderstandings demonstrated on these items.^{Footnote 1}

Following the overview, physics and mathematics results are presented in six subsections that report student performance on the TIMSS and TIMSS Advanced items by grade level, country, gender, and assessment year (see textbox).

*Notes In addition to the separate physics and mathematics sections here, a summary of results across physics and mathematics is provided in Sect. 5.1

We present released example items from TIMSS and TIMSS Advanced to demonstrate specific types of student misconceptions, errors, and misunderstandings along with tabular data showing the percentage of students for each response type by country and on average across the five countries. All example items shown in the report are the standard international version.^{Footnote 2}

In addition to example items, the results include tables and figures that show patterns in the percentage of students demonstrating specific misconceptions, errors, and misunderstandings across countries and grades based on the set of items that measure them.^{Footnote 3} Tree graphs show the female–male difference across countries at each grade level.^{Footnote 4} At grade four and grade eight, separate trend graphs for each country show the percentage of students with misconceptions, misunderstandings, and errors over multiple assessment years.^{Footnote 5}

With the exception of the trend item results, all data reflect the most recent assessment in which each item was administered from 1995 to 2015, which for examples items, is the year the item was released (supplementary materials providing standard errors for all estimates are available for download at www.iea.nl/publications/RfEVol9).

4.2 Physics Results

We selected a set of 16 physics items from the TIMSS and TIMSS Advanced assessments from 1995 to 2015 that measure student understandings and misconceptions related to gravitational force. This item set includes four items from TIMSS Advanced, seven TIMSS items at grade eight, and six TIMSS items at grade four. We identified four performance objectives (POs) measured by these items, each with a specific set of related misconceptions and misunderstandings (Table 4.1).

Table 4.1 Physics performance objectives related to gravity with related misconceptions and misunderstandings, by grade level

Full size table

We provide here a list of the full set of TIMSS and TIMSS Advanced items related to gravity (Table 4.2) organized by performance objective and grade level. This list shows the assessment year(s) when each item was administered, the item format (MC or CR), a brief item description, the figures where the items are shown in the report (released items only), and the specific type(s) of student misconceptions and misunderstandings measured by each item. All physics results reported in this section are based on student performance on this set of items. (See Appendix Table A.1 for additional information on the physics items used in this study, including the specific response options or score categories used to determine the percentage of students demonstrating each type of misconception or misunderstanding.)

Table 4.2 List of TIMSS and TIMSS Advanced physics items related to gravity, organized by performance objective and grade level

Full size table

4.2.1 Student Performance on TIMSS and TIMSS Advanced Items Related to Gravity

The performance of students on the set of gravity items at each grade level covered a broad range both within and across countries (Figs. 4.1, 4.2, and 4.3), with some difficult items (<30% of students were correct) and some easier items (>70% of students were correct).^{Footnote 6} Average item performance across the five countries for TIMSS Advanced ranged from 25 to 54% correct, compared with a range of 36–79% correct at grade eight and 31–78% correct at grade four. Based on the average performance, the range of item percent correct was lower in TIMSS Advanced (25–42% correct) than at grade eight (36–79% correct) or grade four (31–78% correct). Across the set of gravity items at each grade level, at least half of students provided a correct response on five out of seven grade eight items and three out of six grade four items, compared to only one out of the four TIMSS Advanced items. However, there were notable differences in performance patterns observed across the five countries.

Across the four TIMSS Advanced items (Fig. 4.1), the broadest range of item performance was in Slovenia (from 16 to 72% correct) and the United States (from 15 to 64% correct). In contrast, item-level performance ranged from 23 to 49% correct in the Russian Federation and, in Norway, was clustered between 46 and 63% correct. Performance was lowest in Italy, with the item percent correct ranging from 9 to 31%, and generally higher in Norway when considering the full set of items. The most difficult item in all countries was item 2 (“acceleration of a bouncing ball”), a CR item under performance objective 1 (“determine the acceleration due to gravity of thrown objects (after they are thrown)”).^{Footnote 7} In comparison, item 1B (“motion of a ball thrown upward–time between two points”), a CR item from performance objective 2 (“determine the time duration between different points on the path of a thrown object”), was the easiest item for all countries except Italy, with at least 60% of students correct in Norway (63%), the United States (64%), and Slovenia (72%).

At grade eight (Fig. 4.2), a broad range of item performance was found in all five countries and especially in Norway (from 32 to 85% correct). Three MC items, namely items 4, 5, and 6 from performance objective 3 (“determine the effect of gravitational force on moving objects or on objects at rest”), were among the most difficult items in all countries. In particular, performance on item 4 (“gravity acting on a parachute jumper”) ranged from 26% correct in Italy to 30–40% correct in Norway, the Russian Federation, and the United States, to 47% correct in Slovenia. In comparison, the easiest item was item 12 (“direction gravity makes a ball fall at different places on Earth”) from performance objective 4 (“identify the direction of the force due to gravity”), with at least 70% correct in all five countries, and more than 80% correct in Norway, the Russian Federation, and Slovenia.

The broadest range of item performance was found at grade four (Fig. 4.3) in Italy, the Russian Federation, and Slovenia, compared to Norway (where the range was greatest at grade eight) and the United States (where the range was more similar across the three grade levels). In particular, item performance in Slovenia ranged from 16 to 88% correct (a spread of 72 percentage points at grade four). Item 11 from performance objective 3 (“determine the effect of gravitational force on moving objects or on objects at rest”) and items 14 and 16 from performance objective 4 (“identify the direction of the force due to gravity”), were among the most difficult items in all countries, although the specific pattern of performance varied. The CR item 11 (“force causing a marble to roll down a sloping track”) ranged from <20% correct in Italy (15%) and Slovenia (16%), to 29% correct in Norway, to ≥40% correct in the Russian Federation (53%) and the United States (40%). Performance on the MC item 14 (“direction of movement due to gravity”) ranged from 32% correct in Norway to 68% correct in the United States, and performance on MC item 16 (“force that makes objects repel each other”) ranged from 26% correct in Slovenia to 45% correct in the United States. In comparison, item 10 (“force causing an object to fall to the ground”) from performance objective 3 and item 13 (“direction of the force of Earth’s gravity”) from performance objective 4 (both MC items) were amongst the easiest items across all countries, with ≥60% of students responding correctly to both items (except in Slovenia, with 57% correct on item 10). Performance on item 10 was particularly high in the United States (82% correct) and performance on item 13 was particularly high in the Russian Federation (95% correct) and Slovenia (88% correct).

4.2.2 Common Types of Misconceptions and Misunderstandings Related to Gravity Across Countries

A key understanding is that the force due to Earth’s gravity acting on an object on or near Earth’s surface is constant, resulting in a constant acceleration (approximately 10 m s⁻²) directed toward the center of Earth. By the end of secondary school, students are expected to understand that the only forces acting on a thrown object (after it is released) are the downward force due to gravity and air resistance and that the observed motion (slowing, reaching a maximum height, and then falling back down) is the result of the constant acceleration due to gravity at all positions in the path of the object. The first example item (item 1: Fig. 4.4, and Tables 4.3 and 4.4) requires students to apply Newton’s laws of motion to answer two questions about the motion of a ball thrown vertically upward, and shows that many TIMSS Advanced students had difficulty applying these concepts.

Table 4.3 Student performance data for physics item 1A (PA33061A), 2015

Full size table

Table 4.4 Student performance data for physics item 1B (PA33061B), 2015

Full size table

Part A is a MC item requiring students to identify the acceleration of the ball at its highest position (the instant it stops moving upward and reverses direction). A correct response to part A (option D) requires students to know that the acceleration due to gravity is constant and applies equally to the ball at all positions. Across the five countries included in the study, the percent correct ranged from 9% (Italy) to 56% (Norway), with an international average of 42% correct. On average, nearly half of students internationally (48%) indicated that the acceleration was zero (option A), demonstrating the misconception that there is no acceleration since the instantaneous velocity at that position is zero (rather than a constant acceleration due to gravity at all positions). This misconception was less common in Norway (39%), but this still reflects more than one-third of students.

Another 8% of students on average across countries incorrectly determined that the magnitude of the acceleration due to gravity was different at point 3 (either half or twice that at point 2), demonstrating the misconception that the force of gravity changed with the height of the ball (options B and C). Students selecting these options may be confusing gravitational force with gravitational potential energy (PE_g = mgh), which increases with height (h), or incorrectly applying Newton’s law of universal gravitation (F_g = Gm₁m₂/r²), where gravitational force (F_g) decreases with the distance squared (r²). Although this law can be applied to objects that are far from Earth, the difference in force or acceleration due to a change in height is negligible for objects near Earth’s surface, where the acceleration due to gravity (g) is treated as a constant.

In part B, students were asked to determine the time duration between two points on the path of the ball (halfway up and halfway down). A correct response to part B requires students to indicate that the time traveled by the ball is the same on the way up as it is on the way down, as shown in the scoring guide. This relationship can be determined by applying Newton’s laws of motion to the situation where there is a constant acceleration due to gravity (g), and TIMSS Advanced students are expected to have covered this in their physics courses. A common misconception, though, is that the time on the way down is shorter because the ball is accelerating (speeding up) on the way down and decelerating (slowing down) on the way up (i.e., the downward acceleration due to gravity is not treated as constant). On average, slightly more than half (54%) of TIMSS Advanced students answered part B correctly. Most students in Norway (63%), Slovenia (72%), and the United States (64%) provided a correct response. In comparison, about half of the TIMSS Advanced students in the Russian Federation (49%) and about one-fifth in Italy (20%) did so.

In the second TIMSS Advanced example item (item 2: Fig. 4.5 and Table 4.5), students were asked to draw arrows on the figure of a bouncing ball that represent the direction of acceleration of the ball at three positions above the floor (where the only force acting on the ball is due to gravity). Approximately one-quarter of TIMSS Advanced students (25% on average) correctly indicated that the acceleration is directed downward in all three positions (code 10 in the scoring guide).^{Footnote 8} On average, about one-quarter of students (27%) demonstrated the common misconception that the acceleration is in the same direction as the motion of the ball (along the curved path at points P and Q) and that there is no acceleration when the ball is at its maximum height (point R) (code 72 in the scoring guide). This misconception was most common in the United States (38%) and least common in Norway (14%). Another 12% internationally indicated that the acceleration of the ball is upward on its way up (point Q) and downward on its way down (point P), with either an upward or downward acceleration or no acceleration at point R (codes 70 and 71). In addition, about 8% internationally indicated that the acceleration is perpendicular to the direction of motion at points P and Q (code 74). This type of response reflects a misunderstanding that the acceleration of the ball moving along a curved path is caused by a centripetal force directed toward its center (like objects orbiting the Earth). The frequency of this misconception ranged from 1% of students in the United States to 17% of students in the Russian Federation. Another 24% of students on average provided other types of incorrect responses (codes 73 and 79), and about 4% left the item blank.

Table 4.5 Student performance data for physics item 2 (PA13063), 1995

Full size table

In the final TIMSS Advanced item (item 3: Fig. 4.6 and Table 4.6), about three-quarters of students (75% on average across countries) identified gravity as a force acting on a stone after it was thrown straight up in the air (codes 10 and 70 in the scoring guide).^{Footnote 9} The percentage of students who did not identify gravity ranged from almost 3% in Slovenia^{Footnote 10} to 42% in the Russian Federation. These percentages include incorrect responses (codes 71 and 79), as well as students who left the item blank.

Table 4.6 Student performance data for physics item 3 (PA23014), 2008

Full size table

By grade eight, students are expected to be able to determine the effect of gravitational force acting on moving objects or on objects at rest. However, many grade eight students demonstrated the misconception that the force of gravity acts only on falling objects, not on objects that are at rest. This is shown in the first grade eight TIMSS example item (item 4: Fig. 4.7 and Table 4.7), which asks students to identify at which position(s) the force of gravity acts on a parachute jumper (in the aircraft prior to jumping, in freefall, falling with an open parachute, and on the ground after landing). On average across countries, 36% of students correctly identified that the force of gravity was acting on the jumper at all four positions (option D), while over half (57%) indicated that gravity acted only when the jumper was falling (specifically, 45% falling with the parachute open or closed (option B) and 12% in freefall only (option A)). This misconception was common across all five countries, although somewhat less frequent in Slovenia (46%) and more frequent in Italy (68%). The same type of common misconception was demonstrated across countries on two other TIMSS grade eight items involving the application of the same concept in different contexts^{Footnote 11}: a rocket being launched from Earth (item 5); and an apple falling from a tree (item 6). In the context of a rocket launch (item 5), about one-third of grade eight students on average (36%)^{Footnote 12} indicated that gravity acts on the rocket only when it is falling back to Earth and not when it is sitting on the launch pad or moving upward after being launched. In the context of an apple falling from a tree (item 6), 40% of students on average^{Footnote 13} indicated that gravity acts on the apple only while it is falling or still hanging from the tree, and not once it lands on the ground.

Table 4.7 Student performance data for physics item 4 (S032141), 2011

Full size table

The second grade eight example (item 7: Fig. 4.8 and Table 4.8) also showed that many students had a lack of understanding about gravity acting on objects at rest. The item asked students to identify the forces acting on two people sitting on a wall. A complete response (code 10 in the scoring guide) must identify two balanced forces: the downward force due to gravity and the upward force from the wall. On average across the five countries, 63% of students received credit for providing a response that includes gravity and/or the upward force from the wall (codes 10, 11, 12, and 19 in the scoring guide), but only 9% referred correctly to both forces (codes 10 and 11). A complete response was most common in Slovenia (26% of students). On average across countries, about half (51%) of students included only one force, with most of these responses referring to gravity alone. The percentage of students who did not identify gravity (codes 70, 71, 79, and 99) ranged from about a quarter of students in Slovenia (27%) and the United States (26%), to more than half of students in Norway (55%). Some students indicated that there were no forces acting on the people, and others referred to gravity pushing up. These findings at grade eight are similar to results from a previous study of the misconceptions of students in elementary school (Darling 2012). In that study, many students thought that there were no forces acting on a book sitting at rest on a desk.

Table 4.8 Student performance data for physics item 7 (S042211), 2015

Full size table

In the next grade eight example (item 8: Fig. 4.9 and Table 4.9), students were asked to identify the best explanation for why a helium balloon moves upward when it is released. While most students (71% on average across countries) responded correctly that the density of helium is less than the density of air (option A), the most common incorrect response in all countries except Slovenia was that there is no gravity acting on helium balloons (option C), chosen by 14% of students on average across countries.

Table 4.9 Student performance data for physics item 8 (S032281), 2003

Full size table

Another grade eight item (item 9: Fig. 4.10 and Table 4.10) asked students to identify the force that causes a ball thrown upward to fall from its highest point. Gravity causing objects to fall is an expectation even at the grade four level, and most grade eight students (70% on average) correctly identified gravity (code 10 in the scoring guide). However, some grade eight students may have difficulty with the concept that it is gravity alone acting on the ball after it is thrown upward that makes it reverse direction and start falling back down. On average, 30% of grade eight students provided an incorrect response or left the item blank.

Table 4.10 Student performance data for physics item 9 (S042293A), 2015

Full size table

In the final grade eight example (item 12: Fig. 4.11 and Table 4.11), students were asked to identify the direction gravity makes a ball fall at three different places on Earth. While 79% of students on average across countries correctly determined that the ball would fall toward the surface of Earth at all three locations (option D), some students (14% on average) determined that the ball would always fall “down” relative to the bottom of the page rather than toward Earth’s surface (option A). Another 6% of students selected option B or C, both indicating that the ball falls “down” from the position at the “bottom” of the Earth. This indicates a lack of understanding demonstrated by some grade eight students about the direction of gravitational force that is expected by grade four (that gravity pulls objects toward Earth). These results are similar to those from an earlier international study (Sneider and Pulos 1983), which reported that about one-fifth of 13- to14-year-old students (from middle schools in California, USA, and Jerusalem, Israel) demonstrated the misconception that objects fall toward an “absolute down” in space. According to the Sneider and Pulos study, while most students knew that objects fall toward the ground (surface of Earth), only about half understood that this is because the force of gravity is directed toward the center of Earth.

Table 4.11 Student performance data for physics item 12 (S032714), 2003

Full size table

At grade four, students are assessed on their knowledge that gravity is the force that draws objects to Earth. In the first TIMSS grade four example item (item 10: Fig. 4.12 and Table 4.12), about two-thirds of students (66% on average) correctly chose gravity as the force that causes an object to fall (from among a list of given forces). However, some students (15% on average across countries) indicated that it is a push from the hand that causes the object to fall (option D). The percentage of students demonstrating this misconception ranged from 7% in the United States, to 24% in Italy.

Table 4.12 Student performance data for physics item 10 (S031311), 2011

Full size table

In the second grade four example (item 11: Fig. 4.13 and Table 4.13), students were shown an image of a marble placed at the top of a sloping track and asked to name the force that causes the marble to roll down the track. Less than one-third of students on average across countries (31%) correctly named gravity as the force that moves the marble (code 10 in the scoring guide), with performance lowest in Italy (15% correct) and Slovenia (16% correct) and highest in the Russian Federation (53% correct). About half of students (53% on average) provided an incorrect response (code 79), with many students attributing the motion of the marble to another force, such as the wind or a push from the hand, and another 16% left the item blank.

Table 4.13 Student performance data for physics item 11 (S051147), 2015

Full size table

Performance on the next grade four example (item 13: Fig. 4.14 and Table 4.14) showed that while most students (78% on average across countries) correctly indicated that the force of Earth’s gravity acting on a block sitting on a table is in the downward direction (option C), some students (13% on average) demonstrated the misconception that the direction of the force of Earth’s gravity is upward (away from the table top; option A). This misconception was most frequent in Norway (24% of students) and least frequent in the Russian Federation (3%) and Slovenia (4%). An even higher percentage of grade four students (21% on average) indicated that a helium-filled balloon rising in the air is due to an upward push from gravity (item 14, not shown).^{Footnote 14} Both items 13 and 14 demonstrate that some grade four students have the misconception that gravity pushes upward on objects sitting on a solid surface or on objects that are moving upward.

Table 4.14 Student performance data for physics item 13 (S041119), 2015

Full size table

The final two example items at grade four (items 15 and 16: Figs. 4.15 and 4.16) show that some grade four students attributed movement in a direction other than downward (e.g., horizontal movement) to the force of gravity. For item 15 (Fig. 4.15 and Table 4.15), about half of grade four students (51% on average across countries) correctly identified “a boy falling from a tree to the ground” (option D) as an example of an object moving because of the force of gravity. However, many students selected responses where an object moves in a direction other than downward (options A, B, and C). The percentage correct on this item ranged from 30–40% in Norway and Slovenia, to 53% in Italy, to >60% in the Russian Federation and the United States. The most common incorrect response in all countries was “a girl hitting a ball with a bat” (option A), which was selected by 21% of students on average. Similarly, for item 14,^{Footnote 15} 27% of students on average indicated that horizontal movement of objects was due to gravity.

Table 4.15 Student performance data for physics item 15 (S041308), 2007

Full size table

In item 16 (Fig. 4.16 and Table 4.16), many grade four students (43% on average) indicated that gravity can make objects “repel” (or move away from) each other (options A and C). The misconception was common in all five countries, ranging from 37% in the Russian Federation and 38% in Slovenia, to about 44% in Italy and the United States, to 54% of students in Norway. This demonstrates a lack of understanding at grade four that gravity is an attractive force that pulls objects toward Earth.

Table 4.16 Student performance data for physics item 16 (S031313), 2003

Full size table

4.2.3 Patterns in Misconceptions and Misunderstandings Related to Gravity Across Grade Levels and Countries

Student performance data on the individual assessment items described in Sect. 4.2.2 were combined to explore patterns in the percentage of students demonstrating specific misconceptions, errors, and misunderstandings in each country based on the set of items that measure each type of misconception at each grade level (Figs. 4.17, 4.18, and 4.19).^{Footnote 16}

All three of the TIMSS Advanced items were related to the motion of objects thrown upward or bouncing after being thrown. Both item 1A_V1 (“motion of a ball thrown upward—acceleration at highest point,” Fig. 4.4) and item 2 (“acceleration of a bouncing ball,” Fig. 4.5) measure performance objective 1: “determine the acceleration of thrown objects (after they are released).” Based on the frequency of specific response types (Fig. 4.17), many students in all five countries demonstrated the related misconceptions that “objects thrown upward have no acceleration at their maximum height where the instantaneous velocity is zero” (P1B) and that “gravitational acceleration is always in the direction of motion/velocity” (P1C). The other misconception measured by item 1A (V2, options B and C) was “gravitational force (acceleration) acting on objects near Earth’s surface is not constant but changes with the height of the objects above the surface” (P1A). This was far less common, as many more students selected option A (acceleration is zero when the ball is at its maximum height). Across items, the percentage of students demonstrating these types of misconceptions was higher in Italy and lower in Norway than the average across countries.

Performance objective 2 (“determine the time duration between different points on the path of a thrown object”) was measured by item 1B (“motion of a ball thrown upward—time between two points”), and the misconception demonstrated (P2) was quite common in all countries. Item 3 (“forces acting on a stone thrown upward,” Fig. 4.6) measures performance objective 3: “determine the effect of gravitational force on moving objects or on objects at rest.” An incorrect response to this item illustrated a lack of understanding of how the force of gravity acts on objects when they are moving upward, which is related to misconception P3A. This misconception was more common than average in Italy and the Russian Federation and was very infrequent in Slovenia (3% of students). TIMSS Advanced students not identifying gravity in this item may have misconceptions commonly found at the lower grade levels that gravity does not act on objects while they are at rest or moving upward (see Figs. 4.18 and 4.19 for misconceptions about the effect of gravitational force in grade eight and grade four).

Considering the set of TIMSS Advanced items, we noted that the prevalence of specific types of misconceptions differed across countries. In Italy, Norway, and the Russian Federation, misconceptions P1B and P2 were more common, while in the United States, misconception P1C was most common; in Slovenia, misconceptions P1B and P1C were equally common.

The misconceptions held by TIMSS Advanced students that acceleration due to gravity is not constant can arise from related misconceptions about the force of gravity in earlier years. At grade eight (Fig. 4.18), six items measured performance objective 3: “determine the effect of gravitational force on moving objects or on objects at rest.” Five of these items, namely item 4 (Fig. 4.7), item 5 (not shown), item 6 (not shown), item 7 (Fig. 4.8), and item 8 (Fig. 4.9), measured the misconception that “gravity acts only on falling objects, but not on objects at rest or on objects that are moving upward” (P3A). This misconception was very common across countries and, in particular, on item 4 (“gravity acting on a parachute jumper”), where >50% of all students demonstrated the misconception in every country except Slovenia (where this was 46%). The same misconception was measured in the similar item 5 (“gravity acting on a rocket being launched from Earth”) and item 6 (“gravity acting on an apple falling from a tree”). Although the misconception did not appear to be as frequent on these two other items, it was still quite common in all countries, ranging from 31 to 45% on item 5 and from 28 to 48% on item 6. The misconception was also somewhat less common in item 7 (“forces acting on people sitting on a wall”), ranging from 26 to 55% of students. Only 6 to 19% of students demonstrated the misconception that “gravity does not act on objects that are moving upward” (P3A) in item 8 (“why helium balloon moves upward”). The different response patterns across these items in each country may be related to students’ familiarity with the specific contexts. In general, the frequency of misconception P3A at grade eight was less in Slovenia than on average across countries, which was also the case for the TIMSS Advanced item 3, which measured the same type of misconception (Fig. 4.17).

Grade eight item 9 (“force causing a ball thrown upward to fall,” Fig. 4.10), measured the misconception that “gravity alone cannot cause an object initially at rest to start moving; it requires another force/push” (P3B). This misconception, which was common at grade four (Fig. 4.19), was also demonstrated by many grade eight students for this item (Fig. 4.18), with the frequency ranging from 22% of students in the United States to 42% of students in Italy. On average, 30% of grade eight students demonstrated this misconception. This is similar to the average percentage of TIMSS Advanced students on item 3 (25%) (Fig. 4.17) who demonstrated the related misconception P3A by not identifying gravity as a force that acts on a stone after it is thrown straight up in the air. However, there was less variation across countries in the percentage of grade eight students demonstrating the misconception than in the percentage of TIMSS Advanced students.

The last grade eight item, item 12 (“direction gravity makes a ball fall at different places on Earth,” Fig. 4.11), measured performance objective 4: “identify the direction of the force due to gravity.” The misconception demonstrated on this item that “gravitational force causes objects to fall down (in an absolute downward direction in space) rather than toward the center of Earth” (P4A) was less common than the other two types of misconceptions demonstrated by students at grade eight. Misconceptions or misunderstandings related to the direction of the force of gravity, however, were quite common at grade four (Fig. 4.19).

At grade four, two items (10 and 11) measure performance objective 3: “determine the effect of gravitational force on moving objects or on objects at rest.” In both items, incorrect responses demonstrated the misconception that “gravity alone cannot cause an object initially at rest to start moving; it requires another force/push” (P3B), but the frequency of the misconception is quite different on the two items. The misconception was demonstrated by 15% of students on average on MC item 10 (“force causing an object to fall to the ground,” Fig. 4.12), compared to 69% on average on CR item 11 (“force causing a marble to roll down a sloping track,” Fig. 4.13). This illustrates that while most students at grade four demonstrated basic knowledge about the role of gravity in falling objects (item 10), many could not apply this in a less familiar context by connecting gravity to the motion of an object rolling down a sloped surface (item 11).

The remaining grade four items measure performance objective 4: “identify the direction of the force due to gravity.” Item 13 (“direction of the force of Earth’s gravity,” Fig. 4.14) and item 14_V1 (“direction of movement due to gravity,” not shown) measure the misconception that “gravity pushes upward on objects sitting on a solid surface and on objects that are moving upward” (P4B). This misconception was most common in Norway and Italy, and least common in the Russian Federation. Other grade four items measure the related misconception that “gravity can move objects in other directions that are not ‘down’ toward the surface of Earth” (P4C). Item 14_V2 (“direction of movement due to gravity”), item 15 (“example of an object moving due to gravity,” Fig. 4.15), and item 16 (“force that makes objects repel each other,” Fig. 4.16) all measured the misconception that gravity can make objects move in a horizontal direction, and demonstrated a lack of understanding among grade four students that gravity is an attractive force that pulls objects toward the surface of the Earth. The misconceptions on these items were generally common, ranging from 27 to 49% on average. Like misconception P4B, misconception P4C was most common in Norway. This lack of basic understanding at grade four can lead to misconceptions and misunderstandings at higher grade levels, such as those illustrated by items 7 and 12 at grade eight (Fig. 4.18) and by item 2 in TIMSS Advanced (Fig. 4.17).

Across the set of items at grade four, gravity misconceptions were frequently less common than average in the Russian Federation and the United States, and more common than average in Italy, Norway, and Slovenia. This pattern, however, did not persist across the misconceptions at higher grade levels. In particular, the percentages of TIMSS Advanced students demonstrating the gravity misconceptions were generally lower in Norway and higher in the Russian Federation (or not measurably different from the average across countries). The patterns at grade eight were more mixed, with some countries having a higher frequency of some misconceptions and a lower frequency of others. The percentage of students demonstrating gravity misconceptions in Italy was higher than the average for all countries at all three grade levels that were assessed by TIMSS.

4.2.4 Gender Differences in Misconceptions and Misunderstandings Related to Gravity

On average across the five countries, male students outperformed female students on nearly all of the gravity items at all three grade levels (Table 4.17).^{Footnote 17} The only item where there were no significant gender differences in the percentage of students who were correct in any country was grade eight item 12 (“direction gravity makes a ball fall at different places on Earth”). Patterns in performance by gender differed across countries and grades. Gender differences in the percent correct were greatest on the TIMSS Advanced items, with an average male–female difference of at least 10%. In comparison, the average male–female difference in the percent correct ranged from 6 to 12% at grade eight and from 2 to 13% at grade four. In TIMSS Advanced, gender differences were pervasive in Norway, with a significantly higher percentage of males than females responding correctly to all items. In contrast, in Norway, none of the grade eight items, and three of seven items at grade four showed significant gender differences. By comparison, in the United States, there were significant gender differences on all of the grade four items, three of seven items at grade eight, and two of three items in TIMSS Advanced. In Italy, the Russian Federation, and Slovenia, significant gender differences were found for one to three items at each grade level. The specific set of items with significant gender differences varied across countries. None of the items demonstrated significant gender differences in all five countries, but there were two items at each grade level with significant gender differences in three or four countries.

Table 4.17 Performance of female and male students on TIMSS and TIMSS Advanced physics items, by country and grade level, 1995–2015

Full size table

To further understand these gender differences in item performance within and across countries, we compared the percentage of male and female students demonstrating specific types of misconceptions and misunderstandings at each grade level (Tables 4.18, 4.19, and 4.20, and Figs. 4.20, 4.21, and 4.22).^{Footnote 18}

In TIMSS Advanced (Table 4.18 and Fig. 4.20), a higher percentage of female students on average demonstrated misconceptions P1A and P1B on item 1A (“motion of a ball thrown upward—acceleration at highest point”) and misconception P2 on item 1B (“motion of a ball thrown upward—time between two points”). The percentage differences were greatest for misconception P1B, where between 16 and 20% more females than males demonstrated the misconception in Norway, Slovenia, and the United States. In contrast, in Italy 9% more males demonstrated misconception P1B, while 9% more females demonstrated misconception P1A. In the Russian Federation, there was no significant difference in the percentage of males and females demonstrating misconception P1B, but there were 6% more females demonstrating misconception P1A. For misconception P2, between 13 and 16% more females demonstrated the misconception in Italy, Norway, the Russian Federation, and the United States. There were no significant differences found in the percentages of male and female students demonstrating misconception P1C on item 2 (“acceleration of a bouncing ball”) or misconception P3A on item 3 (“forces acting on a stone thrown upward”).

Table 4.18 Percentage of female and male TIMSS Advanced students with misconceptions and misunderstandings about gravity, by country, 1995, 2008, and 2015

Full size table

Table 4.19 Percentage of female and male grade eight students with misconceptions and misunderstandings about gravity, by country, 1995, 1999, 2003, 2011, and 2015

Full size table

Table 4.20 Percentage of female and male grade four students with misconceptions and misunderstandings about gravity, by country: 2003, 2007, 2011, and 2015

Full size table

Compared to TIMSS Advanced, there were fewer significant differences in the percentages of male and female students demonstrating the misconceptions at grade eight (Table 4.19 and Fig. 4.21). Across the five items measuring misconception P3A (“gravity acts only on falling objects, but not on objects at rest or on objects that are moving upward”), there were two items in Italy and the Russian Federation, and only one item in the United States, where a significantly higher percentage of females demonstrated the misconception. In Norway and Slovenia, none of these items showed significant gender differences. The specific set of items with a higher percentage of female students having the misconception varied across countries. On item 4 (“gravity acting on a parachute jumper”), 10% more females than males demonstrated the misconception in both Italy and the Russian Federation, while on item 5 (“gravity acting on a rocket being launched from Earth”), only the Russian Federation showed a gender difference (13% more female than male students demonstrating the misconception). There were no significant gender differences on item 6 (“gravity acting on an apple falling from a tree”). The largest gender difference was seen in Italy for item 7 (“forces acting on people sitting on a wall”), where 21% more female than male students demonstrated the misconception. In the United States, the only significant gender difference was on item 8 (“why helium balloon moves upward”), where 8% more female than male students demonstrated the misconception. The only significant gender difference on item 9 (“force causing a ball thrown upward to fall”) was in Slovenia, where 9% more females than males demonstrated misconception P3B (“gravity alone cannot cause an object initially at rest to start moving; it requires another force/push”). There were no significant differences in the percentages of male and female students demonstrating misconception P4A on item 12 (“direction gravity makes a ball fall at different places on Earth”).

As in grade eight and TIMSS Advanced, there were different patterns of gender differences in misconceptions and misunderstandings across countries on the grade four items (Table 4.20 and Fig. 4.22). Most notably, only in the United States was the frequency of misconceptions and misunderstandings significantly higher for females than males on all items, with the differences ranging from 5 to 11%. Across the two items measuring misconception P3B, significantly more females than males in Italy (11% on item 10), Norway (11% on item 11), and the United States (5% on item 10 and 11% on item 11) demonstrated the misconception that “gravity alone cannot cause an object initially at rest to start moving; it requires another force/push.” On both items measuring misconception P4B (“gravity pushes upward on objects sitting on a solid surface and objects that are moving upward”), only the United States had significantly higher percentages of females demonstrating the misconception (7–8%). The largest gender differences were found on item 14_V2 (“direction of movement due to gravity”), where the percentage of females was significantly higher than the percentage of males demonstrating misconception P4C (“gravity can make objects move in other directions that are not down toward the surface of the Earth”) in Norway (18%), the Russian Federation (9%), Slovenia (17%), and the United States (7%).

4.2.5 Patterns in Misconceptions and Misunderstandings Related to Gravity Over Time

In this section, we present the percentage of students in each country demonstrating a specific type of misconception or misunderstanding over multiple assessment years for the set of trend items from each grade level (Fig. 4.23 and 4.24). For the gravity topic, there were three trend items each in grades four and eight, but no trend items available for TIMSS Advanced. All items except one at grade four (item 11) were administered in three assessment years before they were released.

At grade eight (Fig. 4.23), the trend item data covered assessment years 2003, 2007, 2011, and 2015. There were some significant differences across assessment years in the percentage of students in each country demonstrating the specific types of misconceptions. Item 4 (administered in 2003, 2007, and 2011) and item 7 (administered in 2007, 2011, and 2015) both measure misconception P3A (“gravity acts only on falling objects, but not on objects at rest or on objects that are moving upward”). For MC item 4 (“gravity acting on a parachute jumper”), the percentage of students demonstrating the misconception was not measurably different in 2011 than in the previous two assessments in Italy, Norway, the Russian Federation, and the United States. In contrast, in Slovenia, the percentage of students decreased between 2003 (55%) and 2011 (46%). For CR item 7 (“forces acting on people sitting on a wall”), there were significant differences in Italy and the United States but not in the other countries. In both Italy and the United States, the percentage of students with the misconception in 2011, the second assessment cycle, was higher than in 2015, while the percentage in 2007 was not measurably different. This reflected an 8% decrease in the percentage of students in Italy and a 5% decrease in the United States between 2011 and 2015. The largest differences were found for item 9 (“force causing a ball thrown upward to fall”), which measured misconception P3B (“gravity alone cannot cause an object initially at rest to start moving; it requires another force/push”). In both the Russian Federation and Slovenia, the percentage of students demonstrating the misconception decreased over time, and the difference between the first assessment cycle (2007) and the third (2015) was statistically significant. In Slovenia, this reflected a 10% decrease (from 37 to 27%) and in the Russian Federation, a 12% decrease (from 38 to 26%), with no measurable differences in Italy, Norway, and the United States.

At grade four (Fig. 4.24), the trend item data also covered assessment years 2003, 2007, 2011, and 2015. There were fewer significant differences over time in the percentage of students with misconceptions on the grade four items than at grade eight. As at grade eight, however, the most substantial differences were found in the Russian Federation and Slovenia. Items 10 and 11 both measure misconception P3B (“gravity alone cannot cause an object initially at rest to start moving; it requires another force/push”). On MC item 10 (“force causing object to fall to the ground”), the only countries with a significant difference were the Russian Federation and the United States. In the Russian Federation, the percentage of students demonstrating the misconception was 7% lower in 2011 (19%) than in 2007 (26%). In the United States, the percentage was lower in 2011 (7%) than in 2003 (11%), although the decrease appears to have occurred between 2003 and 2007 (also 7%). On CR item 11 (“force causing a marble to roll down a sloping track”), the percentage of students demonstrating the misconception decreased significantly in the Russian Federation (from 67 to 47%) and Slovenia (from 90 to 84%) between the 2011 and 2015 assessments. Item 13 (“direction of the force of Earth’s gravity”) measures misconception P4B (“gravity pushes upward on objects sitting on a solid surface and on objects that are moving upward”). There was only one statistically significant difference on this item, and that was a slight decrease of 2% of students in the Russian Federation between the 2011 and 2015 assessments (from 5% to 3%).

4.2.6 Summary of Physics Results

In the physics results sections, we have reported on students’ performance on the set of items related to gravity across countries at each grade level (TIMSS Advanced, grade eight, and grade four; see Sect. 4.2.1), patterns in student misconceptions and misunderstandings across countries and grade levels (Sects. 4.2.2 and 4.2.3), gender differences in these misconceptions and misunderstandings (Sect. 4.2.4), and trends over multiple assessment years (Sect. 4.2.5). The frequency of specific types of student misconceptions and misunderstandings at each grade level varied across the five countries included in the study. In each country, and at each grade level, there were some misconceptions and misunderstandings that were demonstrated by at least one-third of students. Gender differences were found at all three grade levels, but were most prevalent on the TIMSS Advanced items. For all trend items (except one in TIMSS Advanced), gender differences favored males, with higher percentages of female students than male students demonstrating the misconception or misunderstanding. Performance on grade eight and grade four trend items administered in multiple assessment years showed that the frequency of certain student misconceptions and misunderstandings decreased over time in some countries but not others.

4.3 Mathematics Results

We selected a set of 29 mathematics items from the TIMSS and TIMSS Advanced assessments from 1995 to 2015 that measured student understandings and errors related to linear equations. This item set included two items from TIMSS Advanced,^{Footnote 19} 19 TIMSS items at grade eight, and eight TIMSS items at grade four. We identified nine performance objectives (POs) related to linear equations that were measured by these items, each with a specific set of related errors and misunderstandings (Table 4.21).

Table 4.21 Mathematics performance objectives related to linear equations with related errors and misunderstandings, by grade level

Full size table

We provide here a list of the full set of TIMSS and TIMSS Advanced items related to linear equations (Table 4.22) organized by performance objective and grade level. This list shows the assessment year(s) when each item was administered, the item format (MC or CR), a brief item description, the figures where the items are shown in the report (released items only), and the specific type(s) of student errors and misunderstandings measured by each item. All mathematics results reported in this section are based on student performance on this set of items. (See Appendix Table A.2 for additional information on the mathematics items used in this study, including the specific response options or score categories used to determine the percentage of students demonstrating each type of error or misunderstanding.)

Table 4.22 List of TIMSS and TIMSS Advanced mathematics items related to linear equations, organized by performance objective and grade level

Full size table

4.3.1 Student Performance on TIMSS and TIMSS Advanced Items Related to Linear Equations

The performance of students on the set of linear equation items across grade levels covered a broad range both within and across countries (Fig. 4.25, 4.26, and 4.27), with some very difficult items (<20% correct) and some easier items (≥60% correct).^{Footnote 20} Average performance across the five countries for the TIMSS Advanced items (Fig. 4.25) ranged from 37 to 43% correct. In comparison, the average item performance on grade eight items (Fig. 4.26) ranged from 13 to 63% correct and on grade four items (Fig. 4.27) ranged from 17 to 62% correct. Some notable differences in performance were observed across the five countries.

For TIMSS Advanced (Fig. 4.25), the widest-ranging item performance was in Norway (from 43 to 62% correct) and the United States (from 40 to 57% correct). In contrast, item-level performance ranged from 36 to 40% correct in the Russian Federation, from 22 to 25% correct in Italy, and from 36 to 39% correct in Slovenia. In three of the five countries, students found item 1B more difficult than item 1A. In contrast, in Italy and Slovenia, 3% more students found item 1A more difficult than item 1B.

At grade eight (Fig. 4.26), a broad range of item performance was found in all five countries, with the lowest range (43 percentage points) in Norway and the highest range (61 percentage points) in the Russian Federation and the United States. The three most difficult items in all countries were CR items. Two of them, items 2 and 3, are from performance objective 2 (“solve system of linear equations in two variables”) and one, item 6, is from performance objective 3 (“interpret the meanings of the slope and y-intercept in linear equations and graphs”). For both items 2 and 3, item performance was lowest in Norway (4% and 1% correct, respectively) and highest in the Russian Federation (36% correct on both items). On item 6, performance was lowest in Italy (2% correct) and highest in the United States (31% correct). In comparison, the two easiest items (item 12 and item 13) were both MC items. Item 12 is from performance objective 4 (“relate algebraic equations to their graphical representations (and vice versa)”) and item 13 is from objective 5 (“write equations to represent situations”). For items 12 and 13, performance was lowest in Norway (44% and 41% correct, respectively) and highest in the United States (78% and 75% correct, respectively).

There was also a broad range of item performance found in each country at grade four (Fig. 4.27). The smallest range in percent correct across grade four items was in the United States (23–66%) and the largest was in the Russian Federation (22–81%). Interestingly, at grade four, the three most difficult items across countries (items 22, 27, and 28) as well as the two of the easiest items (items 23 and 26) were CR items. In comparison, at grade eight, the easiest items were all MC items. One of the most difficult items at grade four, item 22, is from performance objective 7 (“given pairs of numbers in tables or ordered pairs, generate a verbal description of the relationship”). The performance on this item varied from 8% correct in Slovenia to 23% correct in the Russian Federation and the United States. The other two items are from performance objective 9 (“apply algebraic thinking to solve simple real-life problems involving unknowns”). The items that were easy in general for all countries were from performance objective 8 (“given a verbal description of a relationship between a set of numbers, generate pairs of whole numbers that follow that relationship/rule”).

4.3.2 Common Types of Errors and Misunderstandings Related to Linear Equations Across Countries

By the time students reach upper-secondary school, they are expected to be well versed with linear equations/relationships. They should be able to understand solving equations as a process of reasoning and explain their reasoning. The first example item (item 1: Fig. 4.28 and Table 4.23) shows that many students still have difficulty in meeting this expectation. A fully correct score (code 20 in the scoring guide) requires students to correctly answer both parts A and B. A correct response to part A must provide the correct answer with adequate work shown (algebraically or graphically). The algebraic solution includes writing equations for the two different car rental plans (X and Y) and then solving the simultaneous equations to find the point at which they intersect (3000 km) or have the same cost. In part B, a correct response requires students to understand and explain that if the same increase in initial cost is applied to both plans with no other change, the difference between the two y-intercepts remains the same. Therefore, the distance on the x-axis at which the two equations intersect will not change. A common misunderstanding is that the students do not understand the solution well enough to go beyond the set procedure and explain the solution with a change in the initial cost of the rental plan.

Table 4.23 Student performance data for mathematics item 1 (MA33240A and MA33240B), 2015

Full size table

Students received partial credit for providing a correct response to either part A (code 10) or part B (code 11). An incorrect response to part A (code 11, 79, or 99) demonstrates that students cannot evaluate the context, write equations, or apply the correct procedures to solve a system of equations. On average, 58% of students demonstrated this error across the five countries. The percentage of students making this error varied considerably across the five countries, from 79% of students in Italy, to 38% of students in Norway and 43% of students in the United States.

An incorrect response to part B (code 10, 79, or 99) demonstrates that students did not understand the system of equations well enough to explain the impact of the y-intercept change on both equations. That is, they did not demonstrate a deeper understanding beyond applying procedures to solve a pair of linear equations. On average, 64% of students demonstrated this error across the five countries. Italy had the highest percentage of students (76%) demonstrating this misunderstanding, and Norway had the lowest percentage (57%).

By grade eight, students are expected to be able to solve a given system of linear equations in two variables in context as well as not in context. However, many grade eight students made errors in doing so on the TIMSS assessments. This is shown in the first grade eight example item (item 2: Fig. 4.29 and Table 4.24). Similar to the first TIMSS Advanced item (Part A only), this item asks students to write simultaneous linear equations to represent the given situation, and then solve them to get the cost of one pen and two pencils. The important thing is that they need to show their work in order to receive a correct score. On average across the five countries, 25% of students (codes 10 and 11 in the scoring guide) were able to correctly solve this problem and show their work in support of their answers. An additional 3% of students obtained the correct answer but did not show their work (code 70). On average, 72% of students were not able to correctly solve the problem (code 79 and 99 in the scoring guide). The percentage of students not able to complete this task was high across all five countries, with the United States having the lowest percentage (61%) and Italy having the highest (78%).

Table 4.24 Student performance data for mathematics item 2 (M042263), 2007

Full size table

In the second grade eight example (item 3: Fig. 4.30 and Table 4.25), students were expected to solve the system of linear equations provided. Very few students across the five countries could apply the procedure correctly in order to obtain the solution. A high percentage of students either made a mistake (code 79 in the scoring guide) or did not attempt the problem at all (code 99). On average, 85% of students were not able to correctly solve the given system of equations across the five countries; the highest percentage of students unable to solve the problems was in Norway (96%), and the lowest was in the Russian Federation (64%).

Table 4.25 Student performance data for mathematics item 3 (M062237), 2015

Full size table

Another performance objective is that, by the end of grade eight, students should be able to identify and interpret the slope and the intercepts in linear equations shown algebraically and graphically. Item 6 (not shown as it is a secured item) includes two lines on a graph and their equations. Students were expected to determine which line had a greater slope by relating the steepness of the line with the slope of the linear equation. On average across the five countries, nearly 86% of the students failed to correctly identify which slope was larger. In all countries except the United States (69%), more than 85% of the students were not able to correctly relate that the steeper line has the larger slope.

In the next grade eight example (item 9: Fig. 4.31 and Table 4.26), students were expected to identify and select the correct equation of a line based on a verbal description of the conditions given. On average across the five countries, 57% of the students got this item correct, with the lowest percentage correct being in Norway (52%) and the highest in the United States (64%). Two of the distractors (options A and C) were not the equation of a line, and hence could be eliminated. The other distractor (option B) had the intercept and the slope swapped in comparison to the correct equation (option D). On average, 22% of students chose option B; the Russian Federation had the lowest percentage of students choosing option B (11%), while the highest percentage of students choosing this option was in Norway (29%).

Table 4.26 Student performance data for mathematics item 9 (M062074), 2015

Full size table

Another performance objective is that by the end of grade eight, students should be able to translate between algebraic and graphical representations. In item 10 (not shown as it is a secured item), students need to identify the graph of a given equation. On average, across the five countries, only 23% of students correctly identified the graph of the equation given in the item; students in Norway were least likely to get this correct (10%), while the highest percentage of students getting this correct was in the Russian Federation (48%).

Another grade eight example (item 11: Fig. 4.32 and Table 4.27) required students to identify the equation of a line whose graph was given. On average across the five countries, 45% of students got this item correct; Norway had the lowest percentage of correct responses (30%) and Slovenia the highest percentage (55%). In all countries, about half of the students were not able to correctly identify the equation of line for the given graph.

Table 4.27 Student performance data for mathematics item 11 (M042112), 2015

Full size table

In the next grade eight example (item 13: Fig. 4.33 and Table 4.28), students are expected to identify the correct equation of a line from the verbal description of the given situation. More than 60% of students, on average across the five countries, were able to identify the correct equation of the line (option B). Performance on this item varied considerably across countries; Norway had the lowest percentage correct (41%) and the United States the highest percentage correct (75%). The most common incorrect response in all countries was option D, which was selected by 27% of students on average.

Table 4.28 Student performance data for mathematics item 13 (M042202), 2015

Full size table

The next two grade eight example items required students to identify the correct equation of a line from the relationship given in a table. In item 17 (Fig. 4.34 and Table 4.29), all the equations in the response options are for a straight line. On average across the five countries, 46% of the students got this item correct (option C), but performance varied considerably. The percent correct in Norway (28%) and Italy (39%) was significantly lower than the average across the five countries, while that for the Russian Federation (56%) and the United States (62%) was higher than average. Item 19 (Fig. 4.35 and Table 4.30)^{Footnote 21} also required students to identify the correct equation of a line from the relationship given in a table. However, in this case, two response options (options B and C) are not equations for a single straight line. Hence, the choice is between the other two options.

Table 4.29 Student performance data for mathematics item 17 (M032163), 2007

Full size table

Table 4.30 Student performance data for mathematics item 19 (M012046), 1999

Full size table

By the end of elementary school, students are expected to be able to express the relationship between ordered pairs or between two numbers (input/output numbers). The last grade eight example (item 20: Fig. 4.36 and Table 4.31) asked students to identify the correct relationship between the set of given ordered pairs. On average, 56% of the students answered correctly. Performance in Norway (40% correct) and Italy (52% correct) was lower than the international average, while performance in the United States (68% correct) and the Russian Federation (69% correct) was higher than average.

Table 4.31 Student performance data for mathematics item 20 (M012029), 2003

Full size table

At grade four, there are two items (21 and 22) assessing the same performance objective as the previous grade eight item 20. Item 21 (Fig. 4.37 and Table 4.32) is a MC item that required students to correctly identify the verbal description of a rule to obtain a number in column B from a number in column A. On average across the five countries, 45% of students were able to identify the correct description of the rule (option A) from the four choices given. There was a considerable range in performance across countries on this item; in Norway 28% of students responded correctly, while at the other end, in the Russian Federation, 63% of students responded correctly. In item 22 (Fig. 4.38 and Table 4.33), which is a CR item, students were provided with a set of four paired numbers and asked to write a verbal description of the rule applied to the first number in each pair in order to obtain the second number. Student performance on this item was extremely low and there was a high percentage of blank responses (code 99 in the scoring guide). On average across the five countries, only 17% of students were able to provide the correct response. Performance in Italy, the Russian Federation, and the United States was very similar (22–23% correct), and the performance of students in Norway and Slovenia (8–9% correct) was also very similar.

Table 4.32 Student performance data for mathematics item 21 (M041125), 2015

Full size table

Table 4.33 Student performance data for mathematics item 22 (M031227), 2007

Full size table

A related objective at grade four is that students should be able to use a given rule to obtain the output numbers from the input numbers (item 24: Fig. 4.39 and Table 4.34). On average across the five countries, 62% of students were able to provide both correct entries in column B of the table, and another 6% were able to provide one correct entry. An additional 4% of students were able to apply the partial or incomplete rule (i.e., they multiplied the number in column A by 4, but then forgot to add 1 to the result). Student performance on this item varied considerably across countries, covering a range of almost 34 percentage points. Student performance in Norway (47% correct) and Slovenia (57% correct) was below the international average, while that of the United States (66% correct) and the Russian Federation (81% correct) was above the international average.

Table 4.34 Student performance data for mathematics item 24 (M041124), 2015

Full size table

Another example (item 26: Fig. 4.40 and Table 4.35) is from the same performance objective. This problem was placed in a real-life context, and students were expected to complete two table entries in order to receive credit for this item. On average across the five countries, 61% of students were able to complete both tables correctly (code 10 in the scoring guide). An additional 7% of students completed one of the tables correctly (codes 70 and 71). For four countries (Italy, the Russian Federation, Slovenia, and the United States), performance on this item was clustered between 61 and 64% correct. In contrast, in Norway, only 55% of students obtained the correct answer.

Table 4.35 Student performance data for mathematics item 26 (M031242A), 2007

Full size table

The last two grade four example items were problems set in real-life contexts and involved some algebraic thinking, although they did not necessarily require students to write or solve an equation. The first of these two example items (item 27: Fig. 4.41 and Table 4.36) required students to understand the context, find the cost of a child’s ticket, and show their work. The item was worth two score points, and partial credit was given if the answer was correct but no work was shown (code 10 in the scoring guide) or if the response was formulated using the correct method but the answer was not correct due to the student making a computational error (code 11). This was one of the more difficult items for grade four students. On average across the five countries, the weighted percent correct (which takes both full and partial credit into consideration) was only 24, and 66% of students did not receive any credit for this item (codes 70, 79, and 99). Code 70 was given for the responses where students ignored or did not understand the relationship provided for the adult versus child ticket, and simply divided the cost by the number of individuals.

Table 4.36 Student performance data for mathematics item 27 (M031247), 2007

Full size table

In the last item in the set (item 28: Fig. 4.42 and Table 4.37), two relationships are shown in picture format. Students needed to understand the provided relationships between the cost of ice cream cones and lollipops to solve the problem. In part A, they need to find the cost of one ice cream cone and one lollipop together, and in part B the cost of one lollipop. Similar to the previous item, this was also a difficult item, with 25% correct (weighted) on average across the five countries. On average, 16% of students got both parts A and B correct (code 20 in the scoring guide), 10% got only Part A correct (code 10) and 8% got only part B correct (code 11), resulting in a weighted percent correct of 25%. Performance on this item ranged from 20% correct in Italy to 35% correct in Norway, with both being significantly different from the average across the five countries. Performance in the Russian Federation, Slovenia, and the United States was quite similar to each other and to the average across five countries (22–24%).

Table 4.37 Student performance data for mathematics item 28 (M051006), 2015

Full size table

4.3.3 Patterns in Errors and Misunderstandings Related to Linear Equations Across Grade Levels and Countries

Student performance data on the individual assessment items described in Sect. 4.3.2 were combined to explore patterns in the percentage of students demonstrating specific errors and misunderstandings across countries, based on the set of items that measure each type of error or misunderstanding at each grade level (Figs. 4.43, 4.44, and 4.45).^{Footnote 22}

The first performance objective is to interpret the solution to a system of linear equations to answer a question or solve a problem in a real-life context. On average across the five countries, 63% of TIMSS Advanced students (Fig. 4.43) demonstrated misunderstanding M1 (“not able to use slope and intercept to provide an argument in support of the solution to a real-life problem situation”) on item 1B (Fig. 4.28). In the Russian Federation, Slovenia, and the United States, the percentage of students with this misunderstanding was similar to the average. In Italy, the percentage of students demonstrating this misunderstanding was higher (75%) and, in Norway, it was lower (57%) than the average.

Under performance objective 2 (“solve systems of linear equations in two variables”), there are two types of errors, depending on whether students are applying the procedure to a contextualized real-life problem (M2A) or to a non-contextualized problem (M2B). In a contextual situation, students need an additional piece of understanding to evaluate the situation and write the correct equation. This is not needed in the case of a non-contextual situation. There are two example items involving contextualized problems, one from TIMSS Advanced (Fig. 4.28, item 1A) and one from grade eight (Fig. 4.29, item 2). Many students at both grade levels were not able to apply the procedure correctly (error M2A). On average across the five countries, 57% of TIMSS Advanced students and 72% of grade eight students made this error (Figs. 4.43 and 4.44). In Norway, the Russian Federation, and the United States, the percentage of students making error in the procedure was lower in TIMSS Advanced than in grade eight (as might be expected), but in Italy and Slovenia, the percentage of students was comparable at both grade levels. There was more variation across countries in the percentage of students making this error among TIMSS Advanced students (a range of 40 percentage points) than among grade eight students (a range of 17 percentage points). In TIMSS Advanced, the percentage of upper-secondary students not able to apply the procedure ranged from 38% of students in Norway and 43% of students in the United States, to 60% in the Russian Federation and 64% in Slovenia, to 78% in Italy.

The related error (M2B) occurred when students were not able to apply the procedure correctly to solve a non-contextualized problem. There were three items at grade eight, item 3 (Fig. 4.30), item 4 (not shown), and item 5 (not shown), that assessed whether students could correctly apply the procedure to solve a system of linear equations (Fig. 4.44). The average percentage of students making errors in applying the procedure was very high across countries on all three items (84%, 86%, and 62% on items 3, 4, and 5, respectively). As expected, this was more pronounced in the CR items (items 3 and 4) than in the MC item (item 5). The pattern across items was very similar in all counties except the Russian Federation. In the case of the Russian Federation, the difference between CR and MC items was less pronounced (64%, 60%, and 56% on items 3, 4, and 5, respectively).

Another important and prevalent misunderstanding among grade eight students is not being able to relate the steepness of a line with the slope of the line (M3A). The percentage of students demonstrating this misunderstanding on the CR item 6 (not shown) was very high in all countries, with four of the five countries having at least 86% of students in this category. In comparison, 69% of students in the United States demonstrated this misunderstanding (Fig. 4.44).

A related misunderstanding that students demonstrate is confusion between the slope and intercept of an equation (M3B). At grade eight, there are three MC items included in this set that are related to this misunderstanding: item 7 (not shown), item 8 (not shown), and item 9 (Fig. 4.31). For all three items, this specific misunderstanding was tracked by having a distractor in which the two values for intercept and slope were swapped in the equations. This misunderstanding was found to be lower among grade eight students than were some of the other misunderstandings and/or errors (ranging from 10 to 24%, on average). This misunderstanding was less common among students in the Russian Federation (demonstrated by 7–18% of students, depending on the item), but more common in Italy (11–28%), Norway (29–31%), Slovenia (20–24%), and the United States (13–22%).

Performance objective 4 is related to students being able to translate easily between an algebraic equation and the graph of a line. The first misunderstanding related to this performance objective is that students are not able to correctly identify the graph of an equation (M4A). More than 75% of students in Italy, Norway, and Slovenia demonstrated this misunderstanding and were not able to identify the correct graph for a given equation on item 10 (not shown).

A related error is not being able to write or identify the correct algebraic equation or verbal description from the graph of a line (M4B). Students found translating the graph of a line to its algebraic form (item 11: Fig. 4.32) more difficult than translating the graph of a line to a verbal description of the relationship on item 12 (not shown). On average across the five countries, 55% of students were not able to select the correct equation for the given graph of a line (item 11), and 37% of students were not able to select the correct description of the relationship/rule for the given graph of a line (item 12). The percentage of students demonstrating the error (M4B) on item 11 varied from 70% in Norway to 45% in Slovenia. In contrast, students demonstrating the error (M4B) on item 12 varied from 56%, again in Norway, to 22% in the United States. This means students understood the relationship between the two variables but found it difficult to verbalize the relationship in algebraic form. The difference between the error demonstrated on items 11 and 12 was most pronounced in the Russian Federation (20 percentage points) and the United States (37 percentage points) and least pronounced in Italy (5 percentage points).

The next type of misunderstanding (M5) was not being able to translate verbal descriptions into a correct mathematical equation. There were four grade eight items measuring this error: two MC (item 13 in Fig. 4.33 and item 16, not shown) and two CR (items 14 and 15, not shown). As was found with other errors and misunderstandings, the percentage of students demonstrating the misunderstanding was higher on the CR items than on the MC items in all five countries. On all four items, the percentage of students demonstrating the misunderstanding was highest in Norway and lowest in the Russian Federation (except for item 13). On item 13, the percentage of students demonstrating the misunderstanding was lowest in United States.

The related misunderstanding (M6) was not being able to translate the relationship given in a table format into a linear equation. All three grade eight items assessing this misunderstanding were MC in format: item 17 (Fig. 4.34), item 18 (now shown), and item 19 (Fig. 4.35).^{Footnote 23} Across all three items, the misunderstanding was more common among students in Italy than among students in the Russian Federation and the United States. Another related misunderstanding (M7A) is not being able to generate a verbal description given a specific relationship in the form of ordered pairs, which was measured on one grade eight item. The percentage of grade eight students demonstrating this error on item 20 was highest in Norway (60%) and lowest in the Russian Federation and the United States (31% and 32%, respectively).

The next three kinds of errors and misunderstandings were demonstrated at grade four (Fig. 4.45). The first one (M7B) is not being able to generate a verbal description given a relationship in table format. There are three items included in this set (items 21, 22, and 23). Item 21 (Fig. 4.37) and item 23 (not shown) are MC format, and item 22 (Fig. 4.38) is CR format. The misunderstanding appears to be more prevalent on the CR item than on the MC items for all countries. The misunderstanding was more common in Norway (68–91% of students across items) and Slovenia (60–92%) than in Italy (47–78%), the Russian Federation (37–77%), and the United States (36–77%).

Another related error (M8) was not being able to identify a correct set of numbers based on the verbal description of the relationship. There were three grade four CR items that measured this misunderstanding: item 24 (Fig. 4.39), item 25 (not shown), and item 26 (Fig. 4.40). The percentage of students demonstrating this type of error on item 24 covered a range of 34 percentage points across the five countries, from 19% in the Russian Federation to 53% in Norway. Similarly, the percentage of students demonstrating this type of error for item 25 covered a range of 30 percentage points, from 28% in the Russian Federation to 58% in Norway. For item 26, the percentage of students demonstrating this error was more consistent across countries, ranging from 29% in Italy to 35% in Norway.

The last type of error or misunderstanding (M9) was that students were not able to apply algebraic thinking to solve simple real-life problems, which is a precursor skill for linear equations. Both example items 27 (Fig. 4.41) and 28 (Fig. 4.42) were CR in format. In responding to these items, students were not expected to formally write equations but to apply algebraic thinking to solve them. Two-thirds of students on average across the five countries (and at least half in each country) were not able to solve these problems correctly.

4.3.4 Gender Differences in Errors and Misunderstandings Related to Linear Equations

On average across the five countries, there were not many significant gender differences found on the set of items related to linear equations. Patterns in the percent correct by gender (Table 4.38)^{Footnote 24} and the percentage of students with specific errors and misunderstanding by gender (Tables 4.39, 4.40, and 4.41, and Figs. 4.46, 4.47, and 4.48)^{Footnote 25} differed across countries and grade levels. Of the few gender differences observed for linear equation items, more favored males than females.

Table 4.38 Performance of female and male students on TIMSS and TIMSS Advanced mathematics items, by country and grade level, 1999–2015

Full size table

Table 4.39 Percentage of female and male TIMSS Advanced students with errors and misunderstandings about linear equations, by country, 2015

Full size table

Table 4.40 Percentage of female and male grade eight students with errors and misunderstandings about linear equations, by country: 1999, 2003, 2007, and 2015

Full size table

Table 4.41 Percentage of female and male grade four students with errors and misunderstandings about linear equations, by country, 2007, 2011, and 2015

Full size table

Gender differences in percent correct were greatest on the TIMSS Advanced items, with an average female-male difference of at least 8% in favor of boys on both items (Table 4.38). However, this varied across countries, with significant differences found in Italy, the Russian Federation, and Slovenia, but not in Norway or the United States. In comparison, on average across the 19 items at grade eight, significant female-male differences in item percent correct ranged from 4–7%, with two items having measurably different performance that favored boys and two items that favored girls. In all countries, there were two or three grade eight items with significant gender differences. On average at grade four, only one of the eight items had a significant item performance difference that favored males (by 4%). However, in Norway, there were three items where the percent correct favored males.

The specific set of items with significant gender differences varied across countries. None of the items had significant gender differences in all five countries, but the two TIMSS Advanced items had significant gender differences in three countries. In contrast, 12 of the 19 items at grade eight and five of the eight items at grade four had significant gender differences in one or two countries.

Looking at the percent correct in each country, Italy had significant gender differences that favored males on both items in TIMSS Advanced and one item each at grade eight (item 17) and grade four (item 28); grade eight females in Italy performed better on one item (item 7). Norway had one item at grade eight (item 12) and three items at grade four (items 23, 26, and 28) where males performed better than females, and one item at grade eight (item 8) where females performed better than males. The Russian Federation had significant gender differences in favor of males on both items in TIMSS Advanced. In grade eight, there were two items (items 6 and 12) where males performed better than females and two items (items 15 and 19) where females performed better than males. There were no significant gender differences on any grade four items in the Russian Federation. Slovenia had both items in TIMSS Advanced, two items at grade eight (items 3 and 10), and one item at grade four (item 27) where males performed better than females, and one item at grade eight (item 20) where females performed better than males. In comparison, the United States was the only country with no items where male performance was higher; there were two items at grade eight (items 7 and 13) and one item at grade four (item 24) where females did better than males.

Gender differences in the percentage of students demonstrating errors or misunderstandings were greatest for the TIMSS Advanced items (Table 4.39 and Fig. 4.46), with an average female-male difference of 8–9% on items 1A and 1B, ranging from 1 to 19% across countries. Significant differences were found on both items in Italy, the Russian Federation, and Slovenia. In all of these cases, there were higher percentages of females than males with these errors or misunderstandings. There were no statistically significant gender differences on either item in Norway and the United States.

At grade eight (Table 4.40 and Fig. 4.47), 19 items were included in the set. For most of these items, the gender differences in the percentage of students with errors and misunderstandings were found to be not significant. For each country, the items exhibiting a gender difference varied from one item in Norway to five items in the Russian Federation. With the exception of items 7 and 12, gender differences on all other items were found in only one country. For item 7, there were greater percentages of male than female students with the misunderstanding in Italy (a difference of 5%) and the Russian Federation (a difference of 4%). For item 12 (error M4B), there was a greater percentage of females than males with the misconception in Norway and the Russian Federation (percentage differences of 10% and 11%, respectively).

The United States was the only country with significant gender differences on items 8 and 13 (misunderstanding/error M3B and M5), with both items having a higher percentage of males with the misunderstanding/error (5–6%). In contrast, the Russian Federation exhibited gender differences on five of the 19 items in the set. Two items (6 and 12) showed a higher percentage of females in the Russian Federation demonstrating the misunderstanding/error (M3A and M4B), and three items (7, 15, and 19) showed a higher percentage of males demonstrating the misconceptions assessed by these items (M3B, M5, and M6).

Italy exhibited gender differences for two of the 19 items. On item 7, 5% more males than females had the misunderstanding (M3B), while on item 17, 12% more females had the misunderstanding (M6). In Norway, only item 12 had a significant difference, with 10% more females than males demonstrating the error (M4B).

In grade four there were few occurrences of significant gender differences related to errors and misunderstandings across items and countries (Table 4.41 and Fig. 4.48). In Italy and the Russian Federation, there were no significant gender differences on any of the eight items included in the set at grade four. Both Slovenia and the United States had one item each with significant gender differences. In the case of Slovenia, there was a higher percentage of females than males on item 27 (9%) with misunderstanding M9. In contrast, in the United States, a higher percentage of males than females (7%) demonstrated misunderstanding M8 on item 24. Norway is the only country with a significant gender difference on two items (items 23 and 26) showing a higher percentage of females (11% and 14%, respectively) demonstrating the misunderstandings (M7B and M8).

4.3.5 Patterns in Errors and Misunderstandings Related to Linear Equations Over Time

In this section, we present the percentage of students in each country demonstrating a specific type of error or misunderstanding over multiple assessment years for the set of trend items at each grade level (Figs. 4.49 and 4.50). For the linear equations topic, there were 10 trend items at grade eight and seven trend items at grade four, but no trend items available for TIMSS Advanced. At both grades four and eight, there were three items administered in three assessment years before they were released, and all other items were administered for two assessment cycles.

Looking at grade eight (Fig. 4.49), the trend item data covered assessment years 1995, 1999, 2003, 2007, 2011, and 2015. There are some significant differences across assessment years in the percentage of students in each country demonstrating the specific types of errors or misunderstandings. Item 4 (administered in 2011 and 2015) measures error M2B (“not able to apply the procedure correctly to solve non-contextualized problems”). This item shows a decrease of 4% of students making the error in the United States in 2015 in comparison to the previous assessment cycle in 2011.

Item 6 (administered in 2011 and 2015) measures misunderstanding M3A (“not able to relate slope with steepness of lines”), and the trend data for this item show an increase in the percentage of students demonstrating this misunderstanding over time for students in Italy (4%) and the Russian Federation (7%), but a decrease for the United States (9%), Norway (5%), and Slovenia (8%). Item 7 (administered in 1995 and 1999) measures the related misunderstanding M3B (“demonstrates confusion between slope and intercept of an equation”). Trend data are available for only two countries, the Russian Federation and the United States. The percentage of students demonstrating this misunderstanding did not change between 1995 and 1999 for either country.

Item 10 (administered in 2011 and 2015) measures error M4A (“not able to correctly identify the graph of an equation”). The percentage of students making this error decreased by 7% in the United States. Another related error M4B (“not able to translate graphical representations into a mathematical equation or verbal description of a linear relationship”) was measured by item 11 (administered in 2007, 2011, and 2015). The trend data for this item show no change in the percentage of students demonstrating the error from 2011 for all countries. However, for Norway and the United States, the percentage of students demonstrating the misconception increased after the 2007 cycle of TIMSS.

Item 13 (administered in 2007, 2011, and 2015) and item 16 (administered in 2011 and 2015) both measure error M5 (“not able to translate verbal descriptions into a correct mathematical equation”). For item 13 (“formula for K the cost of trip”), the general trend was a decrease in the percentage of students demonstrating this error in 2015 from the previous two assessment years (based on the average percentage across the five countries). However, in Italy and Norway, the percentage appeared to increase between 2007 and 2011 and then decrease between 2011 and 2015 (though the differences were not statistically significant). For item 16 (“set up system of equations”), the difference in the percentage of students did not change between 2011 and 2015.

Item 17 (administered in 2003 and 2007) and item 19 (administered in 1995 and 1999) both measure misunderstanding M6 (“not able to translate relationship shown in table form into a mathematical equation”). Trend data for item 17 show no statistically significant difference in the percentage of students demonstrating this misunderstanding from 2003 to 2007. For item 19, trend data were only available for the Russian Federation and the United States. On this item, the percentage of students demonstrating the misunderstanding decreased between 1995 and 1999 by 10% in the United States and was not statistically different in the Russian Federation.

Item 20 (administered in 1995, 1999, and 2003) measures misunderstanding M7A (“not able to generate a correct verbal description given a specific relationship in the form of ordered pairs”). For this item, complete data for three assessment cycles were only available for the Russian Federation and the United States. In contrast, data for Italy are available for 1999 and 2003 and data for Norway and Slovenia are available for 1995 and 2003. The available data show that the percentage of students with this misunderstanding increased over time in Norway (by 12%) and Slovenia (8%), but decreased in the United States (by 5%).

At grade four, the trend item data covered assessment years 2003, 2007, 2011, and 2015 (Fig. 4.50). Again, at grade four, there were some significant differences over time in the percentage of students demonstrating the misunderstandings. Item 21 (administered in 2007, 2011, and 2015), item 22 (administered in 2003 and 2007), and item 23 (administered in 2003, 2007, and 2011) all measure misunderstanding M7B (“not able to generate a correct verbal description given a specific relationship shown in table form”). In general, across countries the data show a decrease in students having this misunderstanding. Trend data for item 21 show a significant decrease in the percentage of Slovenian students showing the misunderstanding from 2007 to 2015 (by 7%). On item 22, the percentage of students having this misunderstanding decreased from 2003 to 2007 in the Russian Federation (by 15%) and the United States (by 7%). Similarly, in the case of item 23, the percentage of students having this misunderstanding decreased significantly between 2003 and 2011, by 8% in Italy, and by 11% in the Russian Federation and the United States.

Item 24 (administered in 2007, 2011, and 2015) and item 26 (administered in 2003 and 2007) measure misunderstanding M8 (“not able to identify a correct set of numbers that follow a given relationship/rule”). Trend data for item 24 show some interesting patterns over these three points in time. The percentage of students showing this misunderstanding decreased consistently over time in Norway (by 12%), the Russian Federation (16%), and Slovenia (7%). Trend data for item 26 show that the percentage of students with this misconception generally decreased from 2003 to 2007, with significant decreases in Slovenia (by 13%) and the Russian Federation (14%).

Item 27 (administered in 2003 and 2007) and item 28 (administered in 2011 and 2015) measure misunderstanding M9 (“not able to apply algebraic thinking to solve simple real-life problems involving unknowns”). Trend data for item 27 showed that the percentage of students demonstrating the misunderstanding decreased from 2003 to 2007 in the Russian Federation (by 11%) and was not significantly different over time in the other countries. Data for item 28 showed that the percentage of students with this misconception decreased from 2011 to 2015 only for Slovenia (by 6%).

4.3.6 Summary of Mathematics Results

We have reported students’ performance on the set of items related to linear equations across countries at each grade level (TIMSS Advanced, grade eight, and grade four; Sect. 4.3.1), patterns in student errors and misunderstandings across countries and grade levels (Sects. 4.3.2 and 4.3.3), gender differences in these errors and misunderstandings (Sect. 4.3.4), and trends over multiple assessment years (Sect. 4.3.5). The frequency of specific types of student errors and misunderstandings at each grade level varied across the five countries included in the study. In each country, and at each grade level, there were some errors and misunderstandings that were demonstrated by at least 50% of the students. There were some gender differences at all three grade levels. Most of the measurable gender differences favored males (i.e., a smaller percentage of males than females demonstrated the error or misunderstanding measured by the item), but there were some that favored females (primarily at grade eight). Performance on trend items administered in multiple assessment years showed that the frequency of certain student errors and misunderstandings changed over time. Performance on grade eight items showed a decrease for some errors and misunderstandings and an increase for others in some countries. Some measurable decreases were also observed at grade four, but, in contrast to grade eight, there were no items that showed an increase in the percentage of students demonstrating the error or misunderstanding.

Notes

1.
See Sect. 1.2 for definitions of the terminology used throughout the report (performance objectives, misconceptions, errors, and misunderstandings) and how these relate to the physics and mathematics items.
2.
Each country translates the international version of the TIMSS and TIMSS Advanced assessment items into their language(s) of instruction, and these translated national versions are verified by the IEA.
3.
As noted previously, results are based on both released and non-released items that measure the specific types of misconceptions, misunderstandings, and errors. Example items presented in the report are all restricted-use items (from 2015) and released items (from previous assessments). For trend items administered in multiple assessments, the data shown in the tables and figures in this section reflect the most recent assessment year (shown in Tables 4.2 and 4.22).
4.
Gender differences on TIMSS Advanced items are based on the sample of students who took the TIMSS Advanced assessment. As described in Chap. 3, the TIMSS Advanced population reflects a select group of students in each country, and the percentage of female and male students taking more advanced courses may differ from the percentages in the full population of students in their final year of secondary school (see Tables 3.4 and 3.5).
5.
Trend graphs are included only for grades four and eight, as there were no TIMSS Advanced trend items available that measured the specific physics and mathematics concepts under study.
6.
The data (Figs. 4.1, 4.2, and 4.3) reflect the most recent assessment in which each item was administered from 1995 to 2015 (see Table 4.2 for the most recent assessment for each item). Changes in performance between assessment cycles for trend items are reported later (Figs. 4.23 and 4.24).
7.
Item 2 is a released item from the 1995 assessment. Comparable data are not available from Italy. Thus, this item is only included for the other four countries (Norway, the Russian Federation, Slovenia, and the United States).
8.
Item 2 is a released TIMSS Advanced item from 1995. Data for Italy are not available; thus, the international average is based on the other four countries (Norway, the Russian Federation, Slovenia, and the United States).
9.
Item 3 is a released TIMSS Advanced item from 2008. The United States did not participate in the 2008 assessment; thus, the international average is based on the other four countries (Italy, Norway, the Russian Federation, and Slovenia).
10.
This is based on the unrounded data (not shown in Table 4.6). Data to the nearest 0.01% are available at www.iea.nl/publications/RfEVol9.
11.
These two items are not shown as exhibits in the report, but both are released items available from the IEA website (see www.iea.nl).
12.
Item 5 is a released TIMSS item from 1999. Norway did not participate in the 1999 assessment, and comparable data are not available for Slovenia. Thus, the international average is based on the other three countries (Italy, the Russian Federation, and the United States).
13.
Item 6 is a released TIMSS item from the 1995 assessment. Comparable data are not available for Italy. Thus, the international average is based on the other four countries (Norway, the Russian Federation, Slovenia, and the United States).
14.
Item 14 is a secure item from TIMSS 2015 and cannot be shown in the report.
15.
Item 14 is a secure item from TIMSS 2015 and cannot be shown in the report.
16.
The data shown in Figs. 4.17, 4.18, and 4.19 reflect the most recent assessment year, which differs across the set of items at each grade level (from 1995 to 2015). Table 4.2 shows the most recent assessment year for each item.
17.
Table 4.17 displays the percent correct for female and male students on each physics item. The corresponding percent correct of students overall are shown in Figs. 4.1, 4.2, and 4.3.
18.
Tables 4.18, 4.19 and 4.20 display the percentage of female and male students with each misconception in TIMSS Advanced, grade eight and grade four, respectively. The accompanying figures (Figs. 4.20, 4.21, and 4.22) illustrate the differences in the percentages of female and male students at the corresponding grade levels. The corresponding overall percentages of students with the misconceptions are shown in Figs. 4.17, 4.18, and 4.19.
19.
Parts A and B of TIMSS Advanced item 1 are treated as separate items in this report (items 1A and 1B).
20.
The data presented in Fig. 4.25, 4.26, and 4.27 reflect the most recent assessment in which each item was administered from 1995 to 2015. See Table 4.22 for the most recent assessment for each item. Changes in performance between assessment cycles for trend items are reported in Sect. 4.2.5.
21.
Slovenia and Norway did not participate in TIMSS 1999 assessment. Hence, average performance for this item is based on data from three countries (Italy, the Russian Federation, and the United States).
22.
The data shown in Fig. 4.43, 4.44 and 4.45 reflect the most recent assessment year, which differs across the set of items at each grade level (from 1995 to 2015). Table 4.22 shows the most recent assessment year for each item.
23.
Norway and Slovenia did not participate in the 1999 and 1995 assessments.
24.
Table 4.38 displays the percent correct for female and male students for each mathematics item. The corresponding percentages correct for all students are shown in Fig. 4.25, 4.26, and 4.27.
25.
Tables 4.39, 4.40 and 4.41 display the percentage of female and male students with each error or misunderstanding in TIMSS Advanced, grade eight, and grade four, respectively. The accompanying figures (Figs. 4.46, 4.47, and 4.48) provide graphical displays of the differences in the percentage of female and male students at the corresponding grade level. The corresponding percentages of students overall with the errors or misunderstandings are shown in Figs. 4.43, 4.44, and 4.45.

References

Darling, G. (2012). How does force affect motion? Science and Children, 50(2), 50–53.
Google Scholar
Sneider, C., & Pulos, S. (1983). Children’s cosmographies: Understanding the Earth’s shape and gravity. Science Education, 67(2), 205–221.
Article Google Scholar

Download references

Author information

Authors and Affiliations

American Institutes for Research, Washington, DC, USA
Teresa Neidorf, Alka Arora, Ebru Erberber, Yemurai Tsokodayi & Thanh Mai

Authors

Teresa Neidorf
View author publications
You can also search for this author in PubMed Google Scholar
Alka Arora
View author publications
You can also search for this author in PubMed Google Scholar
Ebru Erberber
View author publications
You can also search for this author in PubMed Google Scholar
Yemurai Tsokodayi
View author publications
You can also search for this author in PubMed Google Scholar
Thanh Mai
View author publications
You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Teresa Neidorf .

Rights and permissions

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Reprints and permissions

Copyright information

About this chapter

Cite this chapter

Neidorf, T., Arora, A., Erberber, E., Tsokodayi, Y., Mai, T. (2020). Results for Student Misconceptions, Errors, and Misunderstandings in Physics and Mathematics. In: Student Misconceptions and Errors in Physics and Mathematics. IEA Research for Education, vol 9. Springer, Cham. https://doi.org/10.1007/978-3-030-30188-0_4

Download citation

DOI: https://doi.org/10.1007/978-3-030-30188-0_4
Published: 30 October 2019
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30187-3
Online ISBN: 978-3-030-30188-0
eBook Packages: EducationEducation (R0)

Publish with us

Policies and ethics

Results for Student Misconceptions, Errors, and Misunderstandings in Physics and Mathematics

Abstract

Keywords

4.1 Introduction

4.2 Physics Results

4.2.1 Student Performance on TIMSS and TIMSS Advanced Items Related to Gravity

4.2.2 Common Types of Misconceptions and Misunderstandings Related to Gravity Across Countries

4.2.3 Patterns in Misconceptions and Misunderstandings Related to Gravity Across Grade Levels and Countries

4.2.4 Gender Differences in Misconceptions and Misunderstandings Related to Gravity

4.2.5 Patterns in Misconceptions and Misunderstandings Related to Gravity Over Time

4.2.6 Summary of Physics Results

4.3 Mathematics Results

4.3.1 Student Performance on TIMSS and TIMSS Advanced Items Related to Linear Equations

4.3.2 Common Types of Errors and Misunderstandings Related to Linear Equations Across Countries

4.3.3 Patterns in Errors and Misunderstandings Related to Linear Equations Across Grade Levels and Countries

4.3.4 Gender Differences in Errors and Misunderstandings Related to Linear Equations

4.3.5 Patterns in Errors and Misunderstandings Related to Linear Equations Over Time

4.3.6 Summary of Mathematics Results

Notes

References

Author information

Authors and Affiliations

Corresponding author

Rights and permissions

Copyright information

About this chapter

Cite this chapter

Download citation

Share this chapter

Publish with us

Search

Navigation