Advertisement

Student Achievement and Beliefs Related to Computer and Information Literacy

  • Eveline GebhardtEmail author
  • Sue Thomson
  • John Ainley
  • Kylie Hillman
Open Access
Chapter
Part of the IEA Research for Education book series (IEAR, volume 8)

Abstract

The 2013 International Computer and Information Literacy Study (ICILS) showed that female students demonstrated higher achievement in computer and information literacy (CIL) than male students in 12 of the 14 countries considered, with an average 19 scale points (or one-fifth of a standard deviation) difference across those 12 countries. An analysis of differential item functioning indicated that female students generally performed relatively better on tasks that involved communication, design, and creativity, while male students generally performed relatively better on more technical tasks, and those concerned with security. Female students took a little longer to complete the test than male students; this may have contributed to their better scores. While there were few differences between female and male students’ basic information and communications technologies (ICT) self-efficacy, on average, male students recorded higher specialized ICT self-efficacy than female students in all 14 countries, and the difference was moderate to large in 12 of the 14 countries. General ICT self-efficacy was positively associated with both male and female CIL achievement to a similar extent in all 14 countries. Advanced ICT self-efficacy, however, was less strongly and less consistently related to CIL achievement.

Keywords

Achievement Computer and information literacy (CIL) Differential item functioning Gender differences Information and communications technologies (ICT) International Computer and Information Literacy Study (ICILS) International large-scale assessments Self-efficacy 

3.1 Introduction

As noted in Chap.  1, many large-scale assessments in a range of countries have reported that, on average, female students achieve higher scores than male students on computer, digital, or ICT literacy assessments (the terminology varies but the constructs are similar). These results differ from what might be expected, given the preponderance of males working in information technology and enrolled in computer science courses. These results also differ from the reports of self-reported competencies in the early stages of the introduction of computer technology to school (Cooper 2006; Volman and van Eck 2001). Punter et al. (2017) suggested that there has been a change in the relative performance of female and male students that has accompanied a broader societal change in computer use, from technical to applications incorporating information management and communications that make use of the internet. They argued that the performance of female and male students on different types of task should be investigated. We begin this chapter with an overview of the gender differences reported in the ICILS 2013 international report (Fraillon et al. 2014), and then summarize some detailed analyses of differences between female and male students overall and on different types of task, as well as reported differences in self-efficacy.

3.2 Gender Differences in Overall Performance

As reported in the ICILS 2013 international report (Fraillon et al. 2014), the performance of female students was substantially higher than that of male students in 12 out of the 14 ICILS 2013 countries for which adequate data were collected (Table 3.1). The size of the difference ranged from small in the Czech Republic (12 scale points) to moderate in the Republic of Korea (38 scale points). In the remaining two countries (Thailand and Turkey; in both these countries achievement levels were very low), the differences were negligible.
Table 3.1

Differences in mean performance in computer and information literacy between male and female students

Country

Mean CIL scale score for male students

Mean CIL scale score for female students

Difference in scale scores (males − females)

Republic of Korea

517 (3.7)

556 (3.1)

−38* (4.1)

Slovenia

497 (2.8)

526 (2.8)

−29* (3.6)

Chile

474 (3.9)

499 (3.9)

−25* (4.8)

Australia

529 (3.3)

554 (2.8)

−24* (4.0)

Norway

525 (3.1)

548 (2.8)

−23* (3.5)

Lithuania

486 (3.8)

503 (4.2)

−17* (3.4)

Germany

516 (3.2)

532 (2.9)

−16* (3.8)

Croatia

505 (3.6)

520 (3.1)

−15* (3.5)

Russian Federation

510 (3.4)

523 (2.8)

−13* (2.4)

Slovak Republic

511 (5.1)

524 (4.8)

−13* (4.1)

Poland

531 (3.1)

544 (2.9)

−13* (3.7)

Czech Republic

548 (2.8)

559 (2.0)

−12* (2.7)

Thailand

369 (5.3)

378 (5.7)

−9 (5.6)

Turkey

360 (5.4)

362 (5.2)

−2 (3.8)

Notes Standard errors in parentheses. Because some results are rounded to the nearest whole number, some totals may appear inconsistent. *Significant differences (p < 0.05)

Source Fraillon et al. (2014)

3.3 Gender Differences in Specific Skills

The probability of responding correctly to an item is generally assumed to be dependent only on a student’s ability and not on any other characteristics of the students, such as gender. If an item is easier for a male student than a female student with the same ability, the item is showing differential item functioning (DIF) and will advantage male students in general. The sum of the DIF estimates over all items is zero. The sum of the DIF for certain groups of items may not always add up to zero, however, and can thus reveal that some types of items are easier for male students and others for female students, after taking their ability into account. Items that display large DIFs are usually excluded from the measurement scale during calculation of ability estimates. It is not possible to remove all items that show any DIF, however, and so most remaining items show smaller levels of DIF. DIF values for females were estimated for each of the items in the ICILS 2013 CIL assessment for each of the computer literacy domains/strands, and the estimates over the group of items were summed (Table 3.2).
Table 3.2

Differential item functioning for male and female students by ICILS 2013 strand

Strand

Sum of DIF (female)

Gender DIF favors

Number of items

2.2

Creating information

−1.08*

Females

18

2.1

Transforming information

−0.45

Neither

11

2.3

Sharing information

−0.17

Neither

3

1.2

Accessing and evaluating information

−0.06

Neither

9

1.3

Managing information

0.09

Neither

4

1.1

Knowing about and understanding computer use

0.70

Males

10

2.4

Using information safely and securely

0.97*

Males

10

Total DIF

0.00

Neither

65

Note *DIF estimates > 0.5 of a logit

On average, female students performed better than male students of the same ability when asked to create information and, to a lesser extent, when asked to transform information. Male students outperformed female students of the same ability on items that required knowledge about and understanding of computer use, and on items that concerned using information safely and securely.

These findings agree with those reported in Punter et al. (2017), who examined item bias using different methods; they concluded that overall, ICILS 2013 items showed little gender DIF.

The ICILS 2013 test consisted of three types of items: multiple response items, constructed response items, and large tasks. The large tasks ask students to create an information product, such as a poster, presentation, or website. For instance, students might be asked to use a simple website builder to plan and create a webpage, or to use online database tools to select and adapt information in order to create an information sheet for their peers. DIF was also explored for these item types (Table 3.3). Large tasks were found to be relatively easier for female students. Constructed response and, to a lesser extent, multiple choice items were found to be relatively easier for male students. This pattern was true within each of the domains of CIL.
Table 3.3

Differential item functioning for male and female students by ICILS 2013 item type

Item type

Sum of DIF (female)

Gender DIF favors

Number of items

Large task

−1.72

Females

34

Multiple choice

0.48

Males

7

Constructed response

1.24

Males

24

Total DIF

0.00

Neither

65

Individual assessment items that favored female students generally required skills involving communication, design, and creativity. In comparison, those items favoring male students generally required less creative skills, but more technical skills and greater knowledge of security issues, such as knowing the purpose of a captcha and recognizing spam emails (Table 3.4).
Table 3.4

ICILS 2013 assessment items with a gender differential item functioning estimate favoring male or female students of at least 0.1 of a logit, and the skills required by each item

ICILS 2013 item code

ICILS 2013 test unit

Item difficulty

Gender DIF estimate

Gender DIF favors

Description of skill

S08F

School trip

1.06

−0.17

Females

Create balanced layout of text and images for an information sheet

A10A

After school exercise

−0.37

−0.15

Females

Create an appropriate title design for a poster

B07B

Band competition

−3.01

−0.15

Females

Use software to make an image brighter

A10I

After school exercise

1.03

−0.15

Females

Exclude irrelevant information in a poster

A03Z

After school exercise

−2.84

−0.14

Females

Identify information that is risky to include on a public profile

S08B

School trip

0.14

−0.14

Females

Locate required times on website pages

A10D

After school exercise

0.15

−0.12

Females

Text and background colors contrast to support readability

B09D

Band competition

0.08

−0.12

Females

Text and background colors contrast to support readability

S01Z

School trip

−1.48

0.10

Males

Open a link in a new browser tab

A09Z

After school exercise

−0.31

0.10

Males

Modify the sharing settings of a collaborative document

A02Z

After school exercise

−0.10

0.11

Males

Navigate to a URL given as plain text

B03Z

Band competition

0.13

0.12

Males

Navigate to a text-based URL

S07Z

School trip

2.50

0.15

Males

Interpret and choose a search result based on two criteria

B02Z

Band competition

−0.70

0.16

Males

Explain the features that make one of two passwords more secure

S08D

School trip

0.65

0.17

Males

Convert a description of directions into a visual route on a map

A06C

After school exercise

2.27

0.17

Males

Identify that a link’s URL does not match the URL displayed in the link text

B08Z

Band competition

0.46

0.19

Males

Recognize legal and technical issues associated with image use

A06A

After school exercise

1.70

0.20

Males

Identify that an email does not originate from the purported sender

A04Z

After school exercise

−0.16

0.32

Males

Identify the purpose of a captcha form

Notes Further explanation of the ICILS 2013 item codes, units, item difficulties, and skills required can be found in Fraillon et al. (2014)

3.4 Gender Differences in CIL Self-efficacy

To examine self-efficacy in ICILS 2013, students were asked to report how well they could do each of the following general CIL skills:
  • Search for and find a file on a computer;

  • Edit digital photographs or other graphic images;

  • Create or edit documents (for example assignments for school);

  • Search for and find information needed on the internet;

  • Create a multimedia presentation (with sound, pictures, or video); and

  • Upload text, images, or video to an online profile.

In ICILS 2013, student responses to this set of items were combined into a self-efficacy scale for basic CIL skills. The scale was constructed to have a mean of 50 and a standard deviation of 10.

Female students reported significantly higher levels of general self-efficacy, on average, than male students in six countries (Table 3.5). In Chile and the Republic of Korea, the differences were significant but small, while in the Russian Federation, Croatia, Australia, and Thailand, the differences were negligible (although statistically significant). In the remaining eight countries there were no significant gender differences.
Table 3.5

National averages and gender differences for students’ self-efficacy in basic CIL skills, as reported by students participating in ICILS 2013

Country

National averages for students’ self-efficacy in basic CIL skills

Males

Females

Difference (males − females)

Chile

52 (0.3)

54 (0.3)

−2* (0.3)

Republic of Korea

48 (0.3)

50 (0.3)

−2* (0.3)

Russian Federation

51 (0.3)

52 (0.2)

−1* (0.3)

Croatia

52 (0.3)

53 (0.3)

−1* (0.3)

Australia

51 (0.2)

52 (0.2)

−1* (0.3)

Thailand

39 (0.4)

40 (0.4)

−1* (0.4)

Slovenia

53 (0.3)

54 (0.3)

−1 (0.4)

Slovak Republic

51 (0.3)

51 (0.4)

−1 (0.5)

Norway

52 (0.3)

51 (0.2)

1 (0.3)

Germany

50 (0.3)

49 (0.4)

1 (0.5)

Poland

54 (0.2)

54 (0.3)

0 (0.3)

Czech Republic

51 (0.2)

51 (0.2)

0 (0.3)

Lithuania

49 (0.3)

49 (0.3)

0 (0.4)

Turkey

44 (0.4)

44 (0.5)

0 (0.6)

Notes Standard errors in parentheses. Because some results are rounded to the nearest whole number, some totals may appear inconsistent. *Significant differences (p < 0.05)

Source Fraillon et al. (2014)

Similarly, in ICILS 2013, students were also asked to rate the level of their skills for a set of specialized CIL skills, and a self-efficacy scale for specialized CIL scales was constructed (again with a mean of 50 and a standard deviation of 10). The specialized skills were:
  • Use software to find and get rid of viruses;

  • Create a database (for example, using [Microsoft Access®]);

  • Build or edit a webpage;

  • Change the settings on the computer to improve the way it operates or to fix problems;

  • Use a spreadsheet to do calculations, store data, or plot a graph;

  • Create a computer program or macro (for example, in [Basic, Visual Basic]); and

  • Set up a computer network.

In contrast to the findings for general CIL skills, on average, male students showed higher self-efficacy when rating their ability in specialized CIL skills than female students in all 14 countries (Table 3.6), and the gender differences were much larger. The size of this difference was large in Germany, Norway, the Slovak Republic, the Czech Republic, Poland, Slovenia, and Lithuania, and moderate in Croatia, Australia, Turkey, the Russian Federation, and the Republic of Korea. Only in Chile and Thailand were the differences rated as small.
Table 3.6

National averages and gender differences for students’ self-efficacy in specialized CIL skills, as reported by students participating in ICILS 2013

Country

National averages for students’ self-efficacy in specialized CIL skills

Males

Females

Difference* (males − females)

Germany

51 (0.3)

44 (0.4)

7 (0.5)

Norway

52 (0.3)

46 (0.3)

6 (0.4)

Slovak Republic

54 (0.3)

47 (0.4)

6 (0.5)

Czech Republic

51 (0.3)

45 (0.3)

6 (0.4)

Poland

52 (0.3)

46 (0.3)

6 (0.4)

Slovenia

54 (0.4)

49 (0.3)

5 (0.4)

Lithuania

53 (0.3)

48 (0.3)

5 (0.4)

Croatia

55 (0.3)

50 (0.3)

4 (0.4)

Australia

50 (0.3)

46 (0.2)

4 (0.3)

Turkey

52 (0.4)

48 (0.4)

4 (0.5)

Russian Federation

54 (0.3)

50 (0.3)

4 (0.3)

Republic of Korea

53 (0.2)

50 (0.2)

3 (0.3)

Chile

53 (0.3)

51 (0.4)

2 (0.4)

Thailand

48 (0.4)

46 (0.4)

2 (0.5)

Notes Standard errors in parentheses. Because some results are rounded to the nearest whole number, some totals may appear inconsistent. *All differences were significant (p < 0.05)

Source Fraillon et al. (2014)

In order to examine the association of students’ CIL with ICT self-efficacy beliefs for this report, we computed correlation coefficients for each ICILS country by gender for basic skills (Table 3.7) and for specialized skills (Table 3.8), and calculated Cohen’s d to provide an estimate of the strength of the association. Self-efficacy in basic skills was found to be strongly positively related to achievement for male students in six countries (Australia, Chile, Croatia, the Republic of Korea, the Slovak Republic, and Turkey) and for female students in four countries (the Republic of Korea, Lithuania, the Slovak Republic, and Turkey). In most other countries the association was found to be moderate, while the effect was small for female students in the Czech Republic and Germany. This finding is in contrast to previous studies that have suggested that self-efficacy is not related to performance in CIL (for example, Siddiq et al. 2016).
Table 3.7

Correlation between student self-efficacy for basic skills and CIL achievement, by gender

Country

Correlations between student self-efficacy for basic skills and CIL achievement*

Males

Cohen’s d

Females

Cohen’s d

Australia

0.38 (0.03)

0.8

0.34 (0.03)

0.7

Chile

0.37 (0.03)

0.8

0.32 (0.03)

0.7

Croatia

0.37 (0.03)

0.8

0.30 (0.04)

0.6

Czech Republic

0.24 (0.03)

0.5

0.21 (0.03)

0.4

Germany

0.23 (0.04)

0.5

0.19 (0.04)

0.4

Republic of Korea

0.42 (0.02)

0.9

0.40 (0.03)

0.9

Lithuania

0.35 (0.03)

0.7

0.41 (0.03)

0.9

Norway

0.22 (0.04)

0.5

0.27 (0.03)

0.6

Poland

0.33 (0.02)

0.7

0.34 (0.03)

0.7

Russian Federation

0.30 (0.02)

0.6

0.26 (0.03)

0.5

Slovak Republic

0.36 (0.03)

0.8

0.38 (0.03)

0.8

Slovenia

0.30 (0.03)

0.6

0.24 (0.03)

0.5

Thailand

0.27 (0.03)

0.6

0.32 (0.03)

0.7

Turkey

0.36 (0.04)

0.8

0.38 (0.03)

0.8

Average for all countries

0.32 (0.01)

0.7

0.31 (0.01)

0.7

Notes Standard errors in parentheses. *All correlations were significant (p < 0.05). Effect sizes using Cohen’s d are regarded as insubstantial if d = 0.2, moderate if d = 0.5, and strong if d = 0.8

Table 3.8

Correlation between self-efficacy for specialized skills and CIL achievement, by gender

Country

Correlations between self-efficacy for specialized skills and CIL achievement

Males

Females

Correlation

Cohen’s d

Correlation

Cohen’s d

Australia

0.10* (0.03)

0.2

0.05 (0.03)

0.1

Chile

0.10* (0.03)

0.2

−0.06* (0.03)

−0.1

Croatia

0.18* (0.03)

0.4

0.09* (0.04)

0.2

Czech Republic

0.04 (0.03)

0.1

0.04 (0.03)

0.1

Germany

0.05 (0.03)

0.1

−0.04 (0.04)

−0.1

Republic of Korea

0.20* (0.03)

0.4

0.16* (0.03)

0.3

Lithuania

0.12* (0.03)

0.2

0.09* (0.03)

0.2

Norway

0.01 (0.04)

0.0

−0.05 (0.04)

−0.1

Poland

0.12* (0.03)

0.2

0.04 (0.03)

0.1

Russian Federation

0.08* (0.02)

0.2

−0.02 (0.03)

0.0

Slovak Republic

0.11* (0.04)

0.2

0.06* (0.03)

0.1

Slovenia

0.03 (0.04)

0.1

0.02 (0.03)

0.0

Thailand

0.05 (0.04)

0.1

−0.04 (0.04)

−0.1

Turkey

0.24* (0.04)

0.5

0.17* (0.04)

0.3

Average of all countries

0.10* (0.01)

0.2

0.04* (0.01)

0.1

Notes Standard errors in parentheses. *Correlations were significant (p < 0.05). Effect sizes using Cohen’s d are regarded as insubstantial if d = 0.2, moderate if d = 0.5, and strong if d = 0.8

Self-efficacy in specialized skills, however, was less consistently and less strongly related to CIL achievement (Table 3.8). While a number of the correlations for both male and female students reached statistical significance, the relationship was only found to be of moderate strength for males in Turkey. The strength of the relationship in all other countries was insubstantial.

These differences were noted in the ICILS 2013 international report (Fraillon et al. 2014). The report explains that the finding is not unexpected given the nature of the CIL assessment construct, which is framed around computer and information literacy skills that are not necessarily related to the more technical skills described in the specialized skills construct. Punter et al. (2017) also investigated ICT self-efficacy differences between male and female students, and concluded that the differences may arise as males tend to overestimate their abilities while females tend to underestimate their abilities.

3.5 Gender Differences in Time Taken to Respond to the Test

Another consistent finding in ICILS 2013 across all 14 countries was that male students spent less time responding to the test items, on average, than female students. On average, female students spent one to four seconds longer on each item than male students (Table 3.9).
Table 3.9

Average time in seconds taken to respond per ICILS test item, by gender

Country

Average time (s) for students to respond to test items*

Mean response time males

Mean response time females

Difference (males − females)

Australia

34 (0.4)

37 (0.4)

−3 (0.4)

Chile

35 (0.5)

38 (0.4)

−2 (0.5)

Croatia

36 (0.6)

39 (0.5)

−3 (0.5)

Czech Republic

40 (0.5)

43 (0.4)

−3 (0.4)

Germany

37 (0.6)

40 (0.4)

−4 (0.6)

Republic of Korea

27 (0.5)

31 (0.6)

−4 (0.7)

Lithuania

33 (0.6)

34 (0.6)

−1 (0.4)

Norway

36 (0.5)

39 (0.5)

−3 (0.5)

Poland

39 (0.4)

41 (0.4)

−2 (0.4)

Russian Federation

37 (0.5)

38 (0.5)

−1 (0.4)

Slovak Republic

36 (0.7)

38 (0.5)

−2 (0.4)

Slovenia

35 (0.5)

39 (0.5)

−4 (0.5)

Thailand

31 (0.6)

33 (0.7)

−2 (0.5)

Turkey

23 (0.6)

24 (0.6)

−1 (0.3)

Notes Standard errors in parentheses. *All differences were significant (p < 0.05)

Germany, the Republic of Korea, and Slovenia had relatively higher gender differences in the time taken to respond to items and also higher differences between male and female students’ average performance on the assessment (Table 3.9). Thailand, Lithuania, and the Russian Federation recorded much smaller (though still statistically significant) differences in average response times for male and female students, but varied somewhat in the size of their gender differences in achievement; this was small in Lithuania (17 points) and the Russian Federation (13 points), and non-significant in Turkey (see Table 3.1). These results suggest that response times for items may be a factor in the stronger average performance of female students on the ICILS 2013 CIL assessment. Taking more time to respond to these CIL items may be reflective of more careful and thoughtful responses, rather than being less familiar or less confident in their responses, or needing more time to identify the correct response, as is often the case in other assessments.

3.6 Summary

Research question RQ1 (Sect.  1.4) asked: What is the magnitude of the difference between male and female students in measured computer literacy overall, and for particular types of items?

The findings of ICILS 2013 clearly indicated that, on average, female students achieved higher scores for CIL than male students. This difference was statistically significant in 12 of the 14 countries considered, and averaged 19 scale points (or one-fifth of a standard deviation) across the countries reported here.

Within this overall pattern, we found that differential item functioning analyses indicated that female students generally performed relatively better on tasks that involved communication, design, and creativity skills. In contrast, male students generally performed relatively better on more technical tasks and those concerned with security, such as knowing the purpose of a captcha and recognizing spam emails. In addition, female students took a little longer to complete the test than male students; each item took students an average time of 35 seconds to complete, and female students took between one and four seconds longer to respond to items than male students.

Research question RQ2 (Sect.  1.4) asked: To what extent do female and male students differ in computer self-efficacy overall, and in particular aspects of computing?

We found few differences worthy of note between female and male students’ basic ICT self-efficacy. Differences were significant in only six countries, and of small size in two of these countries. However, on average, male students recorded higher specialized ICT self-efficacy than female students in all 14 countries, and the difference was moderate to large in 11 of the 15 countries. General ICT self-efficacy was positively associated with CIL achievement similarly for both sexes in all 14 countries. Advanced ICT self-efficacy, however, was less strongly and less consistently related to CIL achievement.

References

  1. Cooper, J. (2006). The digital divide: The special case of gender. Journal of Computer Assisted Learning, 22, 320–334. Retrieved from  https://doi.org/10.1111/j.1365-2729.2006.00185.x.CrossRefGoogle Scholar
  2. Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Gebhardt, E. (2014). Preparing for life in a digital age: the IEA International Computer and Information Literacy Study international report. Cham, Switzerland: Springer. Retrieved from https://www.springer.com/gp/book/9783319142210.
  3. Punter, R., Meelissen, M., & Glas, C. (2017). Gender differences in computer and information literacy: An exploration of the performances of girls and boys in ICILS 2013. European Educational Research Journal, 16(6), 762–780. Retrieved from  https://doi.org/10.1177/1474904116672468.CrossRefGoogle Scholar
  4. Siddiq, F., Hatlevik, O. E., Olsen, R. V., Throndsen, I., & Scherer, R. (2016). Taking a future perspective by learning from the past—A systematic review of assessment instruments that aim to measure primary and secondary school students’ ICT literacy. Educational Research Review, 19, 58–84. Retrieved from  https://doi.org/10.1016/j.edurev.2016.05.002.CrossRefGoogle Scholar
  5. Volman, M., & van Eck, E. (2001). Gender equity and information technology in education: The second decade. Review of Educational Research, 71, 613–634. Retrieved from  https://doi.org/10.3102/00346543071004613.CrossRefGoogle Scholar

Copyright information

© International Association for the Evaluation of Educational Achievement (IEA) 2019

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Authors and Affiliations

  • Eveline Gebhardt
    • 1
    Email author
  • Sue Thomson
    • 2
  • John Ainley
    • 3
  • Kylie Hillman
    • 4
  1. 1.ACERCamberwellAustralia
  2. 2.ACERCamberwellAustralia
  3. 3.ACERCamberwellAustralia
  4. 4.ACERCamberwellAustralia

Personalised recommendations