Keywords

1 Introduction

An often-heard concern regarding MOOCs is their high dropout rate [1]. These dropout rates—generally used to assess MOOC-success—are misleading, as often success measurements from traditional education are used [2,3,4,5]. Kalz, Kreijns, Walhout, Castaño-Munoz, Espasa, and Tovar [6] introduced a theoretical framework that combines distal and proximal variables and which takes into account individual intentions and barriers. Since different educational contexts deserve different educational measures [7], Henderikx, Kreijns and Kalz [2] further specified this theoretical framework into a model to take into account individual intentions of MOOC-takers as a starting point for measuring educational success in MOOCs. But, even when taking the individual intentions as a starting point, a study by Henderikx, Kreijns and Kalz [3] showed that there is still a substantial group of MOOC-takers who do not achieve what they intended to do. It seems that they encounter barriers preventing or hindering them from acting out their individual learning intentions.

These barriers can be either MOOC related or non-MOOC related and may cause MOOC-takers to change their individual intentions or even to stop [3]. While there are related studies dealing with the empirical analysis of the effects of barriers to online learning and distance education using various statistical techniques [8,9,10,11,12,13], for the context of massive open online learning such analyses are limited. Current studies on barriers in MOOCs mainly focus on a restricted number of barriers in case studies, qualitative research setups, literature reviews and descriptive studies [14,15,16,17]. There are some studies which empirically investigate barriers to student retention, however these studies merely focus on the effect of specifically selected barriers [18, 19]. Furthermore, some studies in online learning or distance education context grouped types of barriers [9] or aimed to empirically identify barrier components [10]. But, apart from an exploratory study on barriers in MOOCs by Henderikx et al. [3], there is no synthesized overview of MOOC-specific barriers available.

In this study, an exploratory factor analysis was used to categorize these potential barriers and present a MOOC-specific barrier classification, that could contribute to purposefully improve MOOCs and enhance MOOC-taker experiences and intention achievement. First, a literature review will give a brief overview of the most relevant literature on barriers to online learning and MOOCs specifically. Second, the methodology of the study will be reported, followed by the results of the factor analysis. Lastly, the results will be discussed as well as the limitations, implications for practice and recommendations for future research.

2 Literature Review

Many different issues are perceived as possible barriers to online learning and distance education. An extensive literature review on barriers in distance education by Galusha [9] showed that students in a distance learning environment regard financial costs, disruption of family life, lack of support from the employer, lack of feedback, lack of instructor presence, lack of technical assistance, lack of planning assistance, lack of social contact, unfamiliarity with distance learning, lack of computer or writing skills as disablers to their learning. She grouped these barriers into five categories (1) costs and motivators, (2) feedback and teacher contact, (3) student support and services, (4) alienation and isolation and (5) lack of experience and training.

Peltier, Drago and Schibrowsky [12] chose to investigate which role six specific dimensions, drawn from literature, played in perceived effectiveness of online education. These dimensions were (1) instructor support and mentoring, (2) course content, (3) course structure, (4) student-to-student interaction, (5) information technology and (6) instructor-student. Their regression results showed that course content, instructor support and mentoring played a substantial role and can be regarded as the most important barriers - or success factors if positively experienced - to students’ learning experiences.

Other reported challenging characteristics as perceived by students in online learning context are technical problems, perceived lack of community, time constraints and unclear course objectives as found by Song, Singleton, Hill and Koh [13] in their mixed-methods study.

Eom, Wen and Ashill [8] examined the determinants of students’ satisfaction in the context of university online courses. They included the variables course structure, instructor feedback, self-motivation, learning style, interaction, and instructor facilitation, quite similar to the study undertaken by Peltier et al. [12]. Results of the structural equation modelling analysis revealed that instructor feedback and learning style were significant predictors for student success, indicating that these issues are important for learning and could become barriers if students are not satisfied with these specific issues.

Qualitative research by Aragon and Johnson [20] uncovered that self-reported reasons for non-completion of community college online courses were time constraints, lack of instructor interaction, bad course content, lack of communication and technological issues Furthermore, Park and Choi [11] found that lack of family- and work support are positively related to non-completion and can thus be regarded as barriers to online learning.

Research that sought to integrate perceived barriers students (expected to) face in an online distance education context was conducted by Muilenburg and Berge [10]. Their factor analytical study which used the principal component extraction method, revealed that these barriers could be assigned to eight distinctive components: (1) administrative/instructor issues, (2) social interactions, (3) academic skills, (4) technical skills, (5) learner motivation, (6) time and support for studies, (7) cost and access to the internet, (8) technical problems. A composite scores calculation per component identified social interactions as the most important barrier for students’ online-learning. Academic skills have been identified as the least important barrier.

These studies, reporting on aforementioned barriers were all conducted in a general online learning or distance education context. Yet, with the still relatively new online learning environment of MOOCs, research on barriers in MOOC-specific context has caught on and is increasing.

In a study on student retention in MOOCs, Adamopoulos [18] used various text mining and predictive modelling techniques to analyse online student reviews and online available course characteristics. The analysis showed that the negative sentiment for the discussion forum, length of the course and workload had a significant negative effect on student retention. Belanger and Thornton [14] evaluated a MOOC on Bioelectricity by analysing pre- and post-questionnaires and log-data. The main barriers that were mentioned by students as reason for non-completion were time constraints and insufficient background knowledge. A literature review by Khalil and Ebner [15] found, in addition to the barriers mentioned in Belanger and Thornton’s [14] study, that student motivation, feelings of isolation and hidden costs are also considered barriers to MOOC-learning. Further, a descriptive analysis of MOOC data to uncover reasons for dropout by Onah, Sinclair and Boyatt [16], showed that difficulty of the MOOC, timing, lack of digital skills and lack of in-MOOC support were often encountered barriers by MOOC-takers. In addition, Hone and El Said [18] explored factors which affect MOOC retention. Their factor analytic study focused on student experiences with the course instructor, experiences with other learners and experiences with the design features of the course and found that especially instructor interaction and course content are important features for students. If these features are not perceived positively by students, they have the potential to become barriers to their learning and ultimately retention.

Also, a very recent study by Shapiro, Lee, Roth, Li, Çetinkaya-Rundel and Canelas [17] on barriers to retention in MOOCs, sought to identify which antecedents, both inside and outside the course setting, had an impact on MOOC-learning. Their qualitative approach of conducting 36 online interviews identified, in order of severity, lack of time, bad previous experiences, online format and inadequate background as barriers to MOOC-learning.

Previous studies confirmed that research on barriers to learning in MOOCs is developing and has strong parallels with the research findings in online learning and distance education context. Still, a shortcoming of prior studies is that they merely examine several specific potential barriers to MOOC-learning and are limited in their empirical analysis. As it is important to continue to explore potential barriers to MOOC-learning to gain a richer understanding of these issues [17, 21], a next step is to generate a composite overview of potential MOOC-specific barriers or groupings of barriers based on literature and related studies as already available in online learning or distance education context [9, 10].

Henderikx et al. [3], composed an overview of potential barriers based on a limited literature review and made a first effort to categorize these barriers (see Fig. 1).

Fig. 1.
figure 1

Overview of barriers arranged by type [3]

Full size image

The choice for categorization was based on the rationale: which classification would be most useful to MOOC-designers and/or providers and MOOC-takers. The current study took this initial typology of barriers in MOOCs as a starting point. In addition, this overview was expanded by the (potential) barrier items based on findings in the previously discussed literature. An exploratory factor analysis was conducted to empirically summarize the data set and to categorize the barriers.

3 Method

3.1 Participants

The participants were individuals who took part in one or more MOOCs in the Spanish language from different MOOC providers in the last 2 years and who indicated that we could contact them for further research, regardless of whether or not they successfully achieved their personal goals in these MOOCs. 1618 Potential respondents received an invitation to participate in the survey of whom 317 actually completed the survey (163 women, 154 men, Mage = 47, age range: 20–83 years). Most of the participants hold a master (26.1%) or bachelor (32.9%) degree. 8.1% of the participants have a doctorate degree, while 24.8% have an associate or secondary education degree. The remaining 8.1% of the participants finished middle school or below. 66.1% Of the participants are employed for wages, while 13.9% are self-employed. A further 8.5% is currently looking for work and 1.7% is not looking for work. 3.4% of the participants are students, 0.3 military and 6.1% indicated that were retired or other. A majority of the participants participated in up to 5 MOOCs (45.2%). 27.9% participated in 6 to 10 MOOCs, 17% between 11 and 20 MOOCs and 9.9% between 21 and 100 MOOCs. Furthermore, 58.3% of the participants actually finished between 1 and 5 MOOCs, 23.7% finished between 6 and 10 MOOCs, 10.2% between 11 and 20 MOOCs and 7.8% indicated that they finished between 21 and 80 MOOCs. Lastly, 24.4% of the participants prefer the traditional face-to-face way of learning, 39.3% indicates that it makes no difference to them whether they learn face-to-face or online and 36.3% prefers to learn online. Overall, the sample is similar to samples reported in other research on MOOCs [22].

3.2 Materials

A ‘Barriers to MOOC-learning’ survey was developed, which contained items drawn from general online learning, distance education and MOOC-specific context literature on barriers and enablers to learning, as discussed in previous section. After answering several general questions on gender, age, educational background, employment status, MOOC-learning experience and preferred learning context, respondents were asked to indicate to what extent they considered the 44 listed items as barriers to learning in a MOOC on a 5-point Likert scale ranging from ‘to a very large extent’ to ‘not at all’. Examples of items are ‘lack of decent feedback’, ‘family issues’, ‘technical problems with the computer’ and ‘lack of instructor presence’.

3.3 Procedure

Over the course of several weeks potential respondents were invited via email batches using the open source online survey tool Limesurvey (visit http://www.limesurvey.org). Filling out the questionnaire took 5–10 min. After four and six weeks, a reminder was sent to those who did not yet completed the survey.

3.4 Data Screening

The Mahalanobis distance was calculated to identify possible outliers. Based on these calculations, 22 outliers were determined and removed, which resulted in a final sample of 295 cases, which is within the generally accepted item ratio to conduct a factor analysis of 5 to 10 respondents per item [23].

3.5 Analysis

The suitability of the data for factor analysis was assessed by first examining the correlation between items. It was observed that all items correlated with at least .3 with one other item, which is a positive indication of factorability. Additionally, the Kaiser-Meyer-Olkin measure showed a value of .95 which exceeded the recommended minimum value of .6 [24, 25] and the Bartlett’s Test of Sphericity was statistically significant (p < .05), which further supports the factorability of the data. Lastly, the communalities all exceeded .3 (see Fig. 2). Given these indicators, the factorability of the data could be considered positive.

Fig. 2.
figure 2

Component loadings and communalities based on a factor analysis with principal component extraction method and oblimin rotation for 35 items (N = 295)

Full size image

Principal component analysis was selected as extraction method because this method allows for reducing the observed variables to a smaller set of independent composite variables. A cut-off of 0.4 was used for statistical significance of the component loadings and the component structure was examined using both Varimax and Oblimin rotation. After initial analysis, the Oblimin rotation was selected as this rotation method produced the simplest component structure. The Kaiser criterion [26], which retains components with an eigenvalue above 1, and inspection of the screen plot were used to determine the number of components. Yet, as these methods are not considered very accurate [27], parallel analysis was also performed. The first analysis showed the presence of 6 components with eigenvalues above 1, explaining respectively 48,2%, 9,2%, 5,8%, 4,5%, 2,6% and 2,3% of the variance, yet with very few or no loadings in the last two components. The scree plot indicated a break after the 4th component. This was further supported by the results of parallel analysis, which produced 4 random eigenvalues smaller than the first 4 eigenvalues of the PCA. Solutions for 4 and 5 components were then examined, also using Oblimin rotation. The 4-component solution, which explained 67,7% of the variability was preferred because of (a) the combined results of the scree plot and the parallel analyses and (b) the reasonably clear interpretable components.

A total of nine items were removed because they did not meet the criteria of no cross-loading of .4 and failed to have a primary component loading of more than .4, thus not contributing to a simple component structure. The items ‘Procrastinate (delay), cannot get started’, ‘Lack of instructor presence’, ‘Insufficient training/experience to use the delivery system’, ‘Lack of adequate internet access’, ‘Lack of technical assistance’, ‘Technical problems with the site’ and ‘Lack of language skills’ had cross-loadings of more than .4 on multiple components. The items ‘Course content was too easy’ and ‘Course content was too hard’ did not load above .4 on any component. Furthermore, two items which seem very similar: ‘workplace issues’ and ‘workplace commitments’ were not removed as their mutual correlation was low to medium.

For the final stage, a factor analysis of the remaining 35 items, using the principal component extraction method and oblimin rotation was conducted, forcing four components explaining 70,4% of the variance (see Table 1). All items in this analysis had primary loadings over .4 on one single component. The component loading matrix for this final solution is presented in Fig. 2.

Table 1. Total variance explained
Full size table

4 Results

The data analysis indicated that four distinct components summarized the experienced barriers in MOOCs. Component labels were defined that fitted the extracted component/item-combinations. This resulted in the following labels:

Component 1::

Technical and online-learning related skills. MOOC-takers perceived lack of skills like information literacy, insufficient knowledge of the delivery systems, insufficient academic back ground as barrier to MOOC-learning

Component 2::

Social context. These issues are typically related to learning individually. In other words, not learning in a classical and/or physical learning environment. Issues like the impersonal feel of learning, lack of interaction, no collaboration, no interaction and feelings of isolation are included.

Component 3::

Course design/expectations management. This component concerns barriers related to the design and expectations management of the course like the low quality of the course materials, bad course instruction, no instructor interaction, bad course content and lack of feedback

Component 4::

Time, support and motivation. MOOC-takers experience time constraints due to workplace, family and general issues as well as support issues due to lack of family, peer and work support. Further, motivational issues like being responsible for your own learning and motivation are included in this component

As can be seen in Fig. 2, the majority of the commonalities are reasonably high, which indicates that the extracted components represent the variables well.

The internal consistency for each of the components was tested by calculating the Cronbach’s alpha. The alphas were strong: .96 for component 1 (10 items), .882 for component 2 (6 items), .94 for component 3 (9 items) and .94 for component 4 (10 items). Removal of the item ‘prefer to learn in person/face-to-face’ in factor 2, would slightly improve that Cronbach alpha score to .90, yet as the initial score was already strong it was decided not to eliminate this item.

Furthermore, composite scores were calculated for each of the four components (see Table 2), based on the mean of the items that had their primary loadings on each component. Lower scores indicated that this component represented a more severe barrier to the respective MOOC-takers who completed the survey.

Table 2. Means and standard deviations per barrier component and the barrier perceived as most severe (N = 295)
Full size table

5 Discussion

This study has implemented a factor-analytical approach to identify the components that represent the barriers to intention achievement in MOOCs. The iterative process of determining the best fit model, resulted in 4 distinctive components; 1. Technical and online-learning related skills, 2. Social context, 3. Course design/expectations management, 4. Time, support and motivation. This result partly overlaps with a comparable study by Muilenburg and Berge [10], who combined barriers students (expected to) face in an online distance education context into a collective overview for factor analysis. Their analysis found eight components of which administrative issues and costs and access to the internet were not present in our analysis. The lack of barriers concerning administrative issues can be explained by the fact that we did not include administration related barriers in our questionnaire as the administrative issues in MOOCs as a non-formal learning context are not comparable to administrative issues in formal education. An explanation regarding internet issues can most likely be explained by the fact that the Muilenburg and Berge [10] study collected data in 2003. Internet was less available and affordable then compared to present time where access to the internet is inexpensive and available at practically all places and time using various devices.

Also, our study identified one component with technical related issues and online-learning related skill barriers whereas Muilenburg and Berge [10] found three separate components containing technical and academic skills and technical problems. Further, both studies found a social interactions/social context component but time, support and motivation barriers are part of one component in our study, while the Muilenburg and Berge [10] study found two components to cover these barriers. Lastly, our study found one distinct component containing MOOC-design related barriers, which is the largest difference compared to Muilenburg and Berge’s [10] study that found instructor related issues combined with administrative issues in one component. However, this difference could be explained by the fact that, as stated before, we did not include any administrative related barriers in the questionnaire.

The composite scores per barrier component (see Table 3) indicate that course design and time, support and motivation are near enough equally considered as most severe barrier components by the respondents of the barriers to MOOC learning questionnaire. Social context was rated as least severe barrier. In contrast, Muilenburg and Berge’s [10] study found that the social interactions component was perceived as most severe. This is quite a big difference in perception, which might also be explained by the moment in time of the study. As online presence is part of everyday life nowadays, people are increasingly used to this phenomenon; in 2003, this was merely emerging.

Table 3. Classification of barrier components
Full size table

Further, when looking at the course design barrier component, bad course content is rated as most severe barrier. Studies by Peltier et al. [12], and Aragon and Johnson [20], in online learning context, found similar results. In the MOOC-learning context, the study by Hone and El Said [18] also identified course content as an important feature for course retention. Additionally, the most severe barrier included in the time, support and motivation barrier component was lack of time. This is consistent with the findings of Song et al. [13] in online learning context and Belanger and Thornton [14] and Shapiro et al. [17] in MOOC-learning context.

When further assessing the literature review, it stands out that instructor related issues are consistently perceived as important for retention in online learning [8, 12, 20]. Yet, in MOOC-learning context this issue is only found by Hone and El Said [18] and in current study this issue was also not perceived as a severe barrier. This is an interesting observation, even though, with the exception of current study, all of these aforementioned studies merely focused on several specifically selected, mainly course related barriers in their research setup. Possibly, learners have higher expectations, or attach more value to, instructor related issues in a formal education context. As MOOCs are easily accessible and do not have a formal education status (yet), instructor issues, might not be perceived as important for a satisfying learning experience.

An assessment of the barrier components in light of the study by Henderikx et al. [3] resulted in Table 3. From Table 3, it can be inferred that the barrier components and thus the experienced barriers by MOOC-takers are predominantly non-MOOC related. This knowledge can be of value for MOOC-designers and providers. It may guide them in finding suitable re-design solutions or interventions to support MOOC-takers in their learning, even if it concerns non-MOOC related issues. For instance, to support MOOC-takers regarding technical and online-learning related skills, it would be possible to, prior to the start of a MOOC, specifically draw attention to the minimum requirements regarding technical and online learning skills needed to be able to finish the MOOC. The barriers related to social context, that are considered MOOC-related like lack of interaction and lack of collaboration could be addressed in the design of the MOOC by for instance integrating assignments which demand or support interaction and collaboration with fellow MOOC-takers. Course design related barriers are addressable by re-design interventions depending on the specific issues at hand. More-over, barriers concerning time, support and motivation could, even though not MOOC-related, be supported by MOOC providers and/or designers by for instance providing information on how to handle and cope with these kinds of barriers, as well as by providing supporting interventions.

There are some limitations that should be taken into account. Firstly, the sample is limited in the sense that it only considers MOOC-takers who took part in one or more MOOCs in the Spanish language. Future research should replicate this study finding respondents in other MOOC-taker populations. Also, we do not know to what extent the respondents who completed the survey were successful in achieving their personal goals when participating in their respective MOOCs. It would be interesting and potentially valuable to differentiate between these two groups to investigate if either group encounters different barriers. Furthermore, even though the item ratio of 6:1 is within the generally accepted limits for factor analysis (Comrey and Lee 1992), a bigger sample will add to the reliability of the analysis. Further research should be conducted using bigger samples to either confirm or contradict our results. Lastly, as this is the first study examining components influencing intention achievement in MOOCs, further refinement of the barrier overview is necessary. A possible next step is to expand this composed barrier overview into an assessment tool for MOOC-providers and/or designers that can support them in their effort to enhance the MOOC-learning experience, in identifying areas for improvement either MOOC related or not.

To conclude, the aim of this research was to empirically analyse barriers that influence intention achievement in MOOCs and translate this for practical purposes into MOOC or non-MOOC related barrier components. The findings identified 4 barrier components of which the majority contained non-MOOC related barriers, which is useful information for MOOC providers and designers and makes a valuable contribution to the expanding empirical MOOC-research.