Introduction

With the widespread use of the digital devices (computers, tablet computers, and handheld devices) in our daily life, there is an ongoing transition of reading from paperbound to screen-based. Especially, digital natives prefer to read via digital devices rather than paper. As for the digital natives, they are willing to receive information quickly, are skillful at processing information rapidly, are more likely to access information and to make social and professional interactions via various communication technologies (Frand 2000; Prensky 2001a; Oblinger 2003). The thinking patterns and information processing approaches of digital native have fundamentally changed by the new digital devices (Prensky 2001b), such as tablet computers (tablets).

Many studies have been addressed the impacts and effects of reading between paper and digital devices, such as video display terminals (VDT) (Mayes et al. 2001; Noyes and Garland 2008; Wästlund et al. 2005), computer (Noyes and Garland 2008; Yu 2010; Ackerman and Lauterman 2012; Mangen et al. 2013), and e-Readers(Tees 2010; Zambarbieri and Carniglia 2012; Margolin et al. 2013). A previous study has investigated the relationships between the reading comprehension and computer familiarity (Yu 2010). Among these digital devices, the tablets, a new mobile technology, has become a popular educational technology because this kind of technology blends the features of laptops, smartphones, and earlier tablets with always-connected internet and thousands of apps with which to personalize the experience (Johnson et al. 2013). With significantly larger screens and richer gesture-based interfaces than the other mobile technologies, the tablets would be the ideal tools for one-to-one mobile learning and reading.

However, little literature has been done on investigating the effects of reading comprehension between tablets and paper, as well as the relationship between reading comprehension and the familiarity of using tablet computers. Therefore, the present experimental study aims to address the following two issues: (1) Is there any difference on reading comprehension across different media platforms? (2) Does tablet familiarity affect the reading comprehension while reading on tablets? Such research is still in its early stage, but it may have a contribution to lead to a better understanding of the transition of reading behavior from paper-based to digital devices. The results are of great interest both for reading software development and educational technology research.

The next section presents a literature review of the related works on reading comprehension across different media platforms (in particular among tablets, computers and paper) and the works about the relationship between computer familiarity and reading comprehension. In the third section, the research methodology is presented, with full details of the participants, the reading materials, the media platforms, the instruments and procedures used. The results of variance analysis are then presented. The research findings are discussed, and conclusions are drawn in the final section.

Literature review

In this section, the related works regarding reading on different media platforms, computer familiarity, and reading comprehension are introduced in order to provide background information for this study.

Reading on different media platforms

It is obvious that the reading devices emerge in an endless stream. Many studies have been done to investigate the impacts and effects of reading on different media platforms. Some studies focused on the effects of the paper and video display terminals (VDT) (Mayes et al. 2001; Noyes and Garland 2008; Wästlund et al. 2005). Some other studies explored the differences between paper-based reading and computer screen-based reading (Ackerman and Lauterman 2012; Noyes and Garland 2008; Yu 2010; Mangen et al. 2013). Furthermore, almost simultaneously, some researchers compared the difference on reading between e-Reader (such as Amazon Kindle e-Reader) and paper (Tees 2010; Zambarbieri and Carniglia 2012; Margolin et al. 2013). Recently, iPad was introduced as a new member of the reading devices (Zambarbieri and Carniglia 2012; Cheng et al. 2014).

The previous studies revealed that the reading comprehension on an electronic display (such as VDT and computer screen) was poorer than the reading comprehension on paper (Wästlund et al. 2005; Mangen et al. 2013); some other studies reported that the reading comprehension from paper was inferior to VDT or computer screens (Mayes et al. 2001); few studies, however, found there was no significant difference on reading comprehension between VDT and paper-based (Noyes and Garland 2008).

Mayes et al. (2001) found that the comprehension scores of paper reading were lower than those reading from a VDT, and also confirmed that VDT group took significantly longer time reading than paper group. Then, Noyes and Garland (2003) did another experimental study to examine these findings. They found that there were no significant differences on the reading time and number of correct answers, but recall result of the learnt information showed a significant difference.

Wästlund et al. (2005) investigated the influence of VDT and paper presentation of the reading comprehension. They confirmed that the performance in VDT condition was inferior to the performance in the paper condition for both consumption (multiple choice questions) and production (summarization) of information.

Mangen et al. (2013) also explored effects of the technological interface on reading comprehension in Norwegian school. Their study showed that the students who read texts in print scored significantly better on the reading comprehension test than students who read the texts on the computer screen, and participants in the computer reading condition reported higher levels of experienced stress and tiredness than those who reading from paper.

On the other hand, the previous study (Zambarbieri and Carniglia 2012) found there were no significant differences between reading on the mobile devices (such as the e-Reader, e-Ink) and paper in terms of oculomotor behavior. However, mean fixation duration was significantly longer in reading from a computer display than from tablets and other e-Readers. Therefore, consistent conclusions are not drawn on the reading comprehension using different media platforms.

From the above literature, paper-based and computer-based reading methods were still considered for exploring the effects of reading performance. In recent years, tablets have become another common reading devices for teaching and learning because it is a book-like device with great portability, usability, and interactivity. Subsequently, the measurement of reading comprehension is an important way to evaluate reading performance among different media platforms. Therefore, the present study tries to figure out the effects of three media platforms on reading comprehension. The previous studies presented various measurements with different types of formats for reading comprehension. In order to select the proper measurement, the following section will discuss the reading comprehension measurement used in this study.

Reading comprehension: multiple choice and summarization

By knowing reading comprehension from readers, the ability of attaining meaning connected discourse during reading can be identified. The reading comprehension can be classified into two levels, literal level and inferential level (McNamara 2007; Wagner et al. 2009). Literal comprehension, also called the shallow comprehension, is a minimally coherent mental representation which is achieved by readers from the meaning of the explicit knowledge in the text (McNamara 2007). On the other hand, inferential comprehension, namely the deep comprehension, represents a highly coherent, richly integrated, plausible presentation. The readers can use the explicit knowledge in the text and their own prior knowledge to build deeper understanding from the text (McNamara 2007).

Different methods were used to measure the levels of reading comprehension. Some researchers designed the closed-end questions, such as multiple choice questions, to exam students’ literal comprehension. In contrast, in order to examine the inferential comprehension, the open-end questions, such as summarization or short-answer questions, were designed to reorganize the deep process of the text.

As reviewed in the last subsection, studies were conducted to compare the accuracy of the reading comprehension questions (Noyes and Garland 2003; Wästlund et al. 2005), or to examine the performance of the summarization (Ackerman and Lauterman 2012; Yu 2010; Wästlund et al. 2005).

Computer familiarity

With the widespread use of computer-based tests, some studies investigated the relationship between computer familiarity and test scores. Some researchers have developed questionnaires of computer familiarity (Eignor et al. 1998; Kirsch et al. 1998). From those studies, they discovered computer familiarity included four aspects: access, attitudes, experience or use, and related technology. So the questionnaire had four dimensions, including access of where to use computers, self-assessment of attitude and ability, use of and experience with computers, and use of and experience with related technology. The computer familiarity questionnaire in study (Goldberg and Pedulla 2002) surveyed the participants’ familiarity with specific computer hardware and software and the frequency those computer skills were used. Yu (2010) developed the computer familiarity questionnaire (CFQ) with five categories and 33 items. The five categories included assess/availability to computers, attitude to and ability of using computers, with computer-related technology, use of and experience with computers, problem solving when encountering difficulties.

Some studies investigated the relationship between computer familiarity and performance on computer-based test tasks (Taylor et al. 1998; Taylor et al. 1999; Jeong 2014). Goldberg and Pedulla (2002) studied the performance differences according to test mode and computer familiarity on a practice graduate exam. Although tablet computers are popular reading devices now, few studies developed the familiarity for tablets. Zheng et al. (2014) have proposed a new familiarity questionnaire in order to measure tablet familiarity. In this study, we adopted the tablet familiarity to gain student’s familiarity of using tablets and then divide them into different levels of familiarity.

Research methods

In the following subsections, the experiment is described, including a description about how to recruit the participants, what are and how to select the reading materials, media platforms, different levels of tablet familiarity, two questions types of reading comprehension, as well as the procedure of the experiment.

Participants

The participants of this study were 92 second-year college students (20–23 years old, M = 20.47) from one university in Beijing, China. However, data of two students were discarded because they had read the texts before. Data from 90 students were finally used for analysis. All 90 participants were randomly assigned into three groups (paper group, tablet group, and computer group, 30 participants each). They are all Chinese native speakers and have passed the National College Entrance Examination of China in 2012 (NCEE2012). They were recruited by online advertisement for a 60 min paid session in this research.

Materials and instruments

Reading materials

Four reading texts were selected from the bank of Chinese Exam of NCEE 2013. NCEE, short for national college entrance examination, is an academic examination held annually in China. It is a prerequisite for entrance into almost all higher education institutions at the undergraduate level. It is thought to be the most typical and standard test with moderate difficulty, high reliability and validity in China. The participants in this study have participated the NCEE 2012, were regarded as not participating the NCEE 2013. They have not read the texts in Chinese exam of NCEE 2013. In order to provide high validity of reading materials, four experts selected four expository texts with the same difficulty level according to the text’s length, content, and readability. The length of these four texts was about 1,050–1,099 Chinese characters. One of the selected texts was for practical test, and the other three were for the formal test. Text one was about the flying skills of the insects with 1,058 Chinese characters; Text two was about the customary society with 1,050 Chinese characters; Text three was about the born of Lao Tzu and his books with 1,099 Chinese characters; Text four was about the viruses with 1,085 Chinese characters.

Media platform: paper, tablets, and computers

All of the texts were presented via three media platforms. They have the same page layout, including font size, typeface, font color, and line spacing (B5 size, 12 points, Song font, and 1.3 times line spacing). For the paper condition, the passages were printed on B5 paper (176 × 250 mm). For the computer condition, the same passages were presented as PDF-files with Adobe Reader XI for Windows, at 100 % scale. The computer display was 12.5″ LCD monitors operating at 60 Hz, at a resolution of 1,366*768 pixels. For the tablets condition, all passages were presented as ePub files, using iBooks 2.0 for iPad 3. Latin square design was used to balance the effects of procedural treatments.

Tablet familiarity: high, medium, and low levels

Based on the previous study (Zheng et al. 2014), tablet familiarity questionnaire (TFQ) was carried out to measure the participants’ tablet familiarity. Its reliability coefficient was .916. Three levels of the tablet familiarity were calculated by a range of half a standard deviation from the mean score of TFQ in the tablet group. If the score of TFQ is higher than the value of the mean score plus the half a standard deviation, a participant is considered to the high level of tablet familiarity. If the score of TFQ is smaller than the value of the mean score minus the half a standard deviation, a participant is considered to the low level of tablet familiarity. Otherwise, the participant is assigned as the medium level of tablet familiarity.

Reading comprehension: multiple choice questions and summarization

Two question types for reading comprehension, multiple choice questions and summarization, were applied in this study. After the participants read each text, each participant was asked to finish five multiple choice questions which were selected from the NCEE 2013. Each question has four options. For example, one of the questions is “Which one is in accord with the author’s opinion?” The dependent variable of reading comprehension was represented by the participants’ shallow level of reading comprehension and raw scores on this task, with a range of 0–15 points. For the summarization, the participants were asked to summarize with 80–120 Chinese characters by their own words after they read each text. Four experts made the scoring rubric on summarization. The scores of each summarization are ten points, so that the total scores for the summarization are 30 points. After all the participants have finished the summarization task, two of the experts gave the scores for each participant and the average score from the experts represented the summarization result of each participant.

Procedure

The research was composed of the following sessions. Firstly, the participants were told the experiment object and procedure before the experiment and asked to fill out the tablet computer familiarity questionnaire online. Secondly, the participants read Text 1 and finish the task for practice. Thirdly, the participants read the other three texts for test. According to the pilot study (Cheng et al. 2014), we found that all the participants can finish reading each text in 4 min, so all the participants had 4 min to reading each text. After reading each text, the participants were asked to complete five multiple choice questions and the summarization of 80–120 Chinese characters. All the multiple choices test and summarization will be presented and finished with pencil-and-paper.

Results

In order to discover the effects on reading comprehension across different media platforms, a one-way analysis of variance (ANOVA) was calculated on three media platforms to the multiple choice questions and summarization of reading comprehension. The results (as shown in Table 1) revealed that there were significant group differences in the scores of multiple choice questions, F (2, 87) = 5.83, p = .004. Participants in paper group (M = 8.27, SD = 1.91) performed better than tablet group (M = 7.37, SD = 1.73) and computer group (M = 6.53, SD = 2.22). Multiple comparisons indicated that a score of paper group was significantly higher than computer group, p = .004. However, there were no significant differences between paper group and tablet group, p = .214, and a score of tablet group was no significant differences with computer group, p = .265. The results also indicated that there were no significant group differences in the score of summarization, F (2, 87) = 1.81, p = .169.

Table 1 ANOVA summary table for different media platforms and reading comprehension
Full size table

For another research question, in order to find the relationship between reading comprehension and table familiarity in tablet users, a one-way ANOVA was also calculated to compare three levels (low, medium, and high) of tablet familiarity in tablet group with the two question types of reading comprehension. The results in Table 2 showed that there was no significant difference among different levels in the scores of multiple choice questions, F (2, 87) = 0.15, p = .865. However, the results revealed that there were significant group differences in the score of summarization, F (2, 87) = 5.89, p = .008. Participants in high tablet familiarity group (M = 18.58, SD = 2.56) performed better in summarization than medium tablet familiarity group (M = 17.50, SD = 4.77) and low tablet familiarity group (M = 13.93, SD = 2.10). Multiple comparisons showed that the scores of summarization in high tablet familiarity group were significantly higher than low tablet familiarity group, p = .004. No significant differences were found between high and medium levels of tablet familiarity (p = .774) as well as between medium and low levels (p = .068).

Table 2 ANOVA summary table for different tablet familiarity in tablet group and reading comprehension
Full size table

Discussion

The present study is preliminary research on the effects of the different media platforms on reading comprehension and the differences of students using tablets with different levels of the tablet familiarity on reading comprehension can also be seen. From the results, the first major finding was that the scores of the tablet group on multiple choices (shallow level) and summarization (deep level) were higher than computer group and lower than paper group. Only the scores of paper on the shallow level comprehension, however, were significantly higher than the computer-based reading. This confirmed the findings of previous studies (Mangen et al. 2013; Wästlund et al. 2005).

Some studied argued that there may be several possible explanations why participants in paper group scored significantly higher on the shallow level comprehension test (multiple choice questions) than those in the computer group. They can be classified into two categories: device-related [navigation (Mangen et al. 2013), visual fatigue caused by emitting light (Wästlund et al. 2005; Mangen et al. 2013)] and individual differences (cognitive load from the switch between the text media and the test media (Mayes et al. 2001; Noyes and Garland 2008; Mangen et al. 2013) and metacognitive level (Mangen et al. 2013)).

Some studies also found that participants went visual fatigue easily and had poor comprehension because of the emitting light (Noyes and Garland 2003). Tablets, such as iPad, have the same screen technology with computer. The current study found that, however, there was no significant difference between tablets reading and paper reading. It can be eliminated that the visual fatigue caused by the computer screen resulted in the poorer performance of the shallow level comprehension.

During the experiment, all the tests were finished with pen-and-pencil. For computer group, the switch between text media and test media may increase the cognitive load. On the other hand, the participants in tablet group were also under the condition of switching between the reading media and test media. It meant that the cognitive load from the switch between the two media was not the reasons of poorer performance of the shallow level comprehension. This finding was in accord with the results of the previous studies: multiple tasks during reading do not affect the performance (Bowman et al. 2010).

Another potential explanation might be related to individual differences of metacognitive level (Mangen et al. 2013). The present study used a randomized control trial. It was thought that the participants among the three conditions have no difference at the metacognitive level. It could be said that the difference on shallow level comprehension between paper-based reading and computer-based reading was not caused by the difference of metacognitive level.

Mangen et al. (2013) suggested that the way of navigation could affect the process of reading, such as scrolling. Scrolling on a computer screen will lead to spatial instability, which may hamper the reader’s reading comprehension (Mangen et al. 2013). In this study, each text was presented in two pages. Although the participants in computer group were asked to read with single page view, scrolling could not be avoided due to the restriction of screen size. Readers of paper and tablet groups, however, read page by page without scrolling. It might be the main reason that paper reading has better performance than computer reading.

Another finding from the above results was that there was no significant difference of the deep level comprehension across different media platforms. It was likely that this form of test, summarization, was too difficult for the participants. It was hard to test out the differences among the three media platforms. From the frequency analysis, the scores of the summarization closed to a normal distribution. During the experiment, the order of the task was reading the text, do the multiple choice questions, and summarizing. While answering the multiple choice questions, the participants recalled, retold, or reorganized the texts and texts meaning again. This may lead to no difference on summarization among different media platforms.

The variance on reading performance with different media might be affected by the familiarity to the media. In Noyes and Garland’s (2003) research, participants had poor performance while reading on a computer screen than reading on paper because the user experience of computer and paper was totally different. For example, participants had to click the mouse to turn pages on the computer. However, when reading on paper, they just followed their natural reading behavior. Margolin et al. (2013) also argued that college students, which were familiar with computers, may perform better than old population at computer-related tasks. Although tablets had out-breaking growth in the last several years, they were not as popular as computers. The present study investigated the relationship between tablets familiarity and tablets reading comprehension and found that the score of deep level comprehension of high tablet familiarity group was significantly higher than low tablet familiarity group. The results suggested that although tablets have features of computers like LCD screen and internet connectivity, they could also give readers a traditional book-like reading experience with its multi-touch screen and easy-to-use user interface.

Conclusions

Nowadays, reading on screen is part of people’s daily life. With good user experience, tablets have become another popular alternative mobile devices for reading or learning. However, few studies explored the reading performance on tablets, especially reading comprehension across tablets and different media platforms, and reading comprehension between the different levels of tablet familiarity while using tablets. The present study carried out an experiment to investigate the effects of the reading comprehension across paper, tablets, and computers. What is more, tablet familiarity was introduced to figure out the impact of the media familiarity on reading comprehension in this study. The results indicated that the paper group performed significantly better than the computer-based reading on the shallow level comprehension and the high level of tablet familiarity group performed significantly better than low level group on deep level comprehension.

It should be noted that the present study has limitations. The findings and their implications discussed in this paper were based on one study with limited sample size, thus further research with larger data samples is recommended for more generalizable results. The other limitation was the reading texts. In the present research, the reading materials were all expository texts. Maybe there was no difference on reading expository texts. Thirdly, this study used between-subjects design. All participants experienced one of the three conditions. The individual difference could not be measured in the present research.

Once a new technology comes into the classroom, some issues for teaching (difficulty of class management) and learning (cognitive overload, distraction) will emerge. Tablet, obviously, can be a good learning device from the educational perspective, after appropriate training. If providing enough time for teachers and students to adapt the learning devices, these mentioned issues would be things of the past. The results of this study could remove the doubts from the educators, parents, and policy makers, and provide a theoretical foundation for the popularity use of the tablets in education.

The present research was one of the studies of the reading performance on different media platform. Since ICT familiarity might be an important factor of reading performance (Yu 2010), we would like to take it into consideration in our future works. Further, additional variables such as reading material length, reading strategies and reading preference should be examined. Although the current research assessed reading performance by the most common form of assessment: multiple choice questions and summarization, future research should thus use additional measure methods such as observations and questionnaire. Nonetheless, research in this area is still in its infancy, and new research paradigm will need to be developed to help researchers and educators to address the advantages and disadvantages in using new technologies for reading texts.