Keywords

1 Introduction

MOOCs offer learners easy accessible personal development opportunities online, in which they can expand their knowledge on many topics at their own time and pace [1]. These learners each have their individual goals they intent to achieve, which can range from finishing a certain number of modules, following the whole course without doing assignments to completing the course and getting a certificate [2]. However, Henderikx, Kreijns and Kalz [3] found that learners do not always succeed in reaching their individual goals because they encounter barriers to learning. These barriers can be either directly related to the MOOC (e.g. lack of instructor presence, bad course content, lack of instant feedback) or not directly related to the MOOC (e.g. lack of motivation, family issues, technical problems with the pc or internet) and hinder or prevent learners from pursuing their individual learning goals [3, 4]. As barriers seem to have a substantial impact on academic achievement [5, 6], knowledge about factors that affect the encounter of these barriers would be valuable. A recent study by Henderikx, Kreijns and Kalz [7] examined possible factors affecting the encounter of barriers to learning in MOOCs. They found that age (specifically the group between 20–50 years), gender, educational level and previous online learning experience increased the encounter of (certain) barriers in MOOCs.

These factors like age, gender, educational level and previous online learning experience, which have been often studied in various learning contexts (traditional face-2-face contexts, online contexts) specifically in relation to academic achievement, could be classified as “hard” factors as they are visible or quantifiable. “Soft” factors however, which are less tangible and harder to quantify, like achievement emotions also play a part in the encounter of barriers to learning in MOOCs.

Achievement emotions, which can be defined as “emotions tied directly to achievement activities or achievement outcomes” [8], have been identified as very important as they can affect learner engagement, motivation and subsequently academic achievement [9,10,11]. Especially the emotions ‘enjoyment’ and ‘boredom’, which are related to the activity of learning, can be activating (enjoyment) or deactivating (boredom) emotions with regard to learning motivation [11]. Most studies, regarding achievement emotions, were set in traditional face to face settings, but an increasing number of studies investigated the influence of learner emotions in online learning environments [12,13,14]. However, research using Pekrun et al.’s [10] enjoyment and boredom scales in MOOC settings is sparse. With this paper, we provide two contributions to the field of learning in open online environments: (1) We examine the enjoyment and boredom scales in a MOOC context and (2) we investigate whether there is an association between enjoyment or boredom and the extent to which learners experience barriers in MOOCs.

2 Theoretical Background and Related Work

2.1 Barriers to Learning in MOOCs

Learning and succeeding in open non-formal learning environments like MOOCs can be challenging [15]. A reason why learners do not always reach their personal learning goals is that they encounter barriers to learning, which prevent or hinder them in their learning progress. These barriers can be either MOOC-related or non-MOOC related [3, 4]. Typical MOOC-related barriers often mentioned by learners are lack of interaction [5, 16, 17], lack of instructor presence [17] and course content [5, 18]. Non-MOOC related barriers experienced by learners are insufficient academic knowledge [6, 19, 20], lack of time [5, 6, 19, 21]. A principal component analysis study by Henderikx et al. [4], distinguished four different barrier components and interestingly found that most barriers could be classified as non-MOOC related (see Table 1).

Table 1. Classification of barrier components [4]
Full size table

Furthermore, the participants in this study indicated that own responsibility for learning, lack of time, bad course content, lack of motivation, low quality of instruction and/or materials and family issues were most often considered as barriers.

2.2 Achievement Emotions

Achievement emotions or academic emotions are essential to understand as they can affect a learner’s effort, motivation to persist and strategies for learning [11]. According to Pekrun and Linnenbrink-Garcia [11] achievement emotions can be either related to the activity of learning i.e. activity-related emotions such as enjoyment or boredom or related to the outcome i.e. outcome-related emotions such as hope and pride. These emotions can occur in class-related, learning-related and test-related environments and can be divided into positive and negative, activating and deactivating emotions with respect to their impact on student performance and motivation (see Table 2; [22]).

Table 2. Overview achievement emotions adapted from Pekrun [23]
Full size table

For example, enjoyment is regarded as an activity-related positive activating emotion as it is found to enhance effort and motivation [23]. Boredom, on the other hand, is considered an activity-related negative deactivating emotion while it undermines effort and motivation [23]. For the purpose of this study we solely focus on enjoyment and boredom as emotions during learning. As mentioned in the introduction, the majority of the studies about achievement emotions are set in traditional face to face (offline) contexts.

One of the major differences between offline and online learning environments is the lack of interaction between the learner and the instructor when it comes to identifying and responding to the emotional state of the learner [24]. Emotions are a reaction to the environment and thus in the case of limited interaction with an instructor, these emotions are mainly fuelled by the online learning environment [24]. Dillon et al. [13] explored self-reported emotions learners experience while learning in MOOCs. During the runtime of a MOOC, learners were asked, at seven different occasions, to indicate their feelings about the content (i.e. video’s, assessments) by picking 2 emotions. These emotions were in part selected from Pekrun et al. [10]. The results of their study indicated that emotions are content sensitive, meaning that different content evoked different emotions. Furthermore, positive emotions, more specifically enjoyment, were the most often experienced emotions with regard to the various contents and negative emotions affected dropout and thus learner achievement. A study by Tze, Daniels, Buhr and Le [35] has analysed the connection between achievement in MOOCs and emotions. Results show that learners with low levels of boredom and low levels of guilt are more likely to deal with the course material and the instructional logic of the course design.

A recent study by Beirne, Mac Lochlainn and Mhichíl [25], examined self-reported learner emotions in a MOOC. During the 3-week runtime of the MOOC, learners were prompted 6 times per week at various points in the MOOC to indicate their emotions they experienced at that moment in time. Similar to Dillon et al. [13], they found that positive emotions were predominant during the course and that certain content evoked negative emotions which may affect achievement. In addition, a meta-analysis on achievement emotions in technology based learning environments implies that levels of emotions differ across learning environments but that the effect of emotions generally supported the hypotheses that positive emotions like enjoyment are positively related to achievement and that negative emotions like boredom are negatively related to achievement [26]. Furthermore, their meta-analysis also indicated that there is no exclusive effect between gender and positive or negative emotions. These findings indicate that the interaction with the content in MOOCs as opposed to the interaction with an instructor in offline learning environments, recalls positive as well as negative emotions and affects learner achievement regardless of gender.

Nevertheless, our aim was to explore a possible relationship between emotions (enjoyment and boredom) and barriers experienced by learners in MOOCs, as opposed to individual achievement. As online learning in a MOOC is very different from offline classroom learning, regarding time, pace, location and intellectual support [27] boredom might affect the experience of barriers, as this activity-related deactivating emotion was found to activate avoidance motivation [28] and “…triggers impulses to escape the situation” [23, p. 533]. In other words, learners who experience higher levels of boredom may feel the impulse to escape the situation and may therefore be more susceptible to experiencing barriers to learning. Therefore, we expected that boredom is positively associated with the experience of barriers. Learners who experience more boredom also experience barriers to learning in MOOCs as more severe. For enjoyment we expect to find the opposite direction, learners who experience higher levels of enjoyment will experience barriers while learning in MOOCs as less severe. In addition, while enjoyment and boredom are known to affect learner motivation [10, 11], we again predict a negative association between enjoyment and a positive association between boredom and motivation related barriers i.e. ‘procrastination’, ‘motivation’ and ‘responsibility for one’s own learning’ specifically.

3 Method

3.1 Participants

Sample 1.

The participants were individuals who took part in the MOOC on Marine Litter developed by UN environment and the Open University. The MOOC aimed at stimulating leadership and offered opportunities for actionable and change-oriented learning, related to marine litter [29]. It ran from May until July 2017, covering eight blocks for eight weeks by providing not only in-depth knowledge, but also useful tools and instruments in addressing marine litter problems. In total 39 participants completed the survey (26 women, 13 men, Mage = 42,15 years, age range: 25–68 years).

Sample 2.

The participants were individuals who took part in one or more MOOCs in the Spanish language from different MOOC providers and on different topics in the last 2 years and who indicated that we could contact them for further research, regardless of whether or not they successfully achieved their personal goals in these MOOCs. 1618 Potential respondents received an invitation to participate in the survey of whom 299 actually completed the survey (146 women, 153 men, Mage = 47,02 years, age range: 20–83 years). The samples were added together for the purpose of analysis.

3.2 Materials

Achievement Emotions.

The achievement emotions were measured using the respective scales of the Achievement Emotions Questionnaire developed by Pekrun et al. [10]. Participants were asked to refer to the most recent MOOC they participated in when answering these questions related to enjoyment and boredom. As achievement emotions are context specific, it is important to differentiate between academic settings [10]. In addition, achievement emotions may refer to before, during or after learning or to the outcome [11]. For this reason, we focused on the learning related emotions enjoyment and boredom and more specifically only used the items referring to enjoyment (6 items, e.g. ‘I enjoy acquiring new knowledge’) and boredom (9 items, e.g. ‘The material bores me to death’) during learning (the items referring to before and after learning were thus excluded). The items were slightly adjusted to fit the learning context of MOOCs and scored on a five-point Likert scale, 1 = totally disagree and 5 = totally agree.

These scales have been validated in multiple studies in traditional face to face contexts [10], however as the online learning context fundamentally differs from the traditional learning context, the construct validity of both scales was tested using principal axis factoring with direct oblimin rotation. The factorability of the data was supported by the Kaiser-Meyer-Olkin measure that showed a value of .93 [the recommended minimum value is .6; 30, 31] and the Bartlett’s Test of Sphericity which was statistically significant (p < .05). The number of factors was determined by combining the results of the scree-plot which indicated a break after the 2nd factor, the eigenvalues exceeding 1 and the parallel analysis, which produced 2 random eigenvalues smaller than the first 2 eigenvalues of the principal axis factoring analysis. These two factors explain 79,6% of the variance. Furthermore, all items had primary loadings well exceeding the cut-off point of .4 on one single factor. In addition, the standardized factor loadings were mainly between .8 and 1, which can be regarded as excellent quality loadings [32] and showed a very high internal consistency per factor (see Table 3). These indices all point towards a good construct validity.

Table 3. Factor analysis and scale reliability of enjoyment and boredom
Full size table

Barriers.

A ‘Barriers to MOOC-learning’ survey was developed, which contained items drawn from general online learning, distance education and MOOC-specific context literature on barriers and enablers to learning, as discussed in previous section. After answering several general questions about gender, age, educational background, employment status, MOOC-learning experience and preferred learning context, respondents were asked to indicate to what extent they considered the 44 listed items as barriers to learning in a MOOC on a 5-point Likert scale ranging from 1 = to a very large extent’ to ‘not at all’. Examples of items are ‘lack of decent feedback’, ‘family issues’, ‘technical problems with the computer’ and ‘lack of instructor presence’.

3.3 Procedures

Sample 1.

In week 5 of the MOOC on Marine Litter, participants were invited to complete a survey, via a link in the MOOC, about barriers to learning in the MOOC which also included questions about the experience of enjoyment or boredom during learning. Participation was voluntary and filling out the questionnaire took 5–10 min.

Sample 2.

Over the course of several weeks potential respondents were invited via email batches using the open source online survey tool Limesurvey (visit http://www.limesurvey.org) to complete a survey about barriers to learning in MOOCs which also included questions about the experience of enjoyment or boredom during learning. Participation was voluntary and filling out the questionnaire took 5–10 min. After four and six weeks, a reminder was sent to those who did not yet completed the survey.

3.4 Data Screening

The Mahalanobis distance was calculated to identify possible outliers. Based on these calculations, 34 outliers were determined and removed. Due to the high number of outliers we ran the analyses twice, with and without outliers, to verify whether it would influence the analyses. Yet no difference in outcomes was detected. The final sample of included 304 cases, which exceeds the generally accepted item ratio to conduct a factor analysis of 5 to10 respondents per item [33].

4 Results

The relationships between the four different barrier components as determined by Henderikx et al. [4], the motivation specific barriers and the achievement emotions enjoyment and boredom [10] were investigated using Pearson product-moment correlation coefficient. Table 4 shows the associations between the 4 barrier components and enjoyment and boredom. A small statistically significant correlation was found between boredom and the barrier components 1, 2 and 4, indicating that learners who experience higher levels of boredom experience barriers related to ‘technical and online learning related skills’, ‘social context’ and ‘time, support and motivation’ more severe. Enjoyment was not statistically significantly correlated to any of the barrier components.

Table 4. Correlations between achievement emotions and barrier components (N = 304)
Full size table

Table 5 displays the correlation results of the analysis between the motivation related barriers ‘procrastination’, ‘motivation’ and ‘responsibility for one’s own learning’ and enjoyment and boredom. The results indicate a small statistically significant correlation between boredom and each of the motivation related barriers and a small significant correlation between enjoyment and the barrier ‘own responsibility for learning’. Learners with higher levels of boredom experience each of the motivation related barriers more severe and learners with higher levels of enjoyment experience the barrier ‘own responsibility for learning’ less severe.

Table 5. Correlations between achievement emotions and motivation related barriers (N = 304)
Full size table

While the findings of Loderer, Pekrun and Lester [26] did not find an exclusive effect between gender and positive or negative emotions, we were interested in whether a difference in association between the variables could be detected for male and female learners. Table 6 shows that, for males, a small statistically significant correlation was found between boredom and the barrier components 1, 2 and 4 and a small statistically significant correlation between enjoyment and the barrier component 4. Men who experience higher levels of boredom experience barriers related to ‘technical and online learning related skills’, ‘social context’ and ‘time, support and motivation’ more severe. In addition, men with higher levels of enjoyment experience the barriers related to ‘time, support and motivation’ less severe. For females, a small statistically significant correlation was detected between boredom and the barrier components 2 and 4, indicating that females who are more bored experience barriers related to ‘social context’ and ‘time, support and motivation’ more severe. No statistically significant correlation was found between female enjoyment and any of the barrier components.

Table 6. Correlations between achievement emotions and barrier components by gender
Full size table

The results in Table 6 showed that for both men and women a small statistically significant correlation was found between boredom and the barrier components 2 and 4. To test whether the difference between these correlations is statistically significant the observed value of z (zobs) was calculated. The zobs for the established correlations between boredom and barrier component 2 and 4 are respectively 0.362313 and 0.646987. These values are both within the range of −1.96 < Zobs < 1.96, which indicates that there is no statistical difference in the strength of the correlation between boredom and barrier components 2 and 4 for males and females.

Furthermore, examining the results of the analysis by gender of the motivation related barriers and enjoyment and boredom in Table 7, it can be inferred that for both males and females a small statistically significant correlation between boredom and each of the motivation related barriers was found. Men as well as women with higher levels of boredom experience each of the motivation related barriers more severe. For neither men nor women an association between enjoyment and the motivation related barriers was found.

Table 7. Correlations between achievement emotions and motivation related barriers by gender
Full size table

Again, the zobs was calculated to establish whether the difference between the found correlations for men and women was statistically significant. The zobs for the correlations between boredom and the motivation related barriers ‘procrastination’, ‘motivation’ and ‘responsibility for one’s own learning’ were respectively 0.629734, −0.26742 and −0.094891. Similar to previous found scores, these values are all within the range of −1.96 < zobs < 1.96, and thus indicate that there is no statistical difference in the strength of the correlation between boredom and each of the motivation related barriers for males and females.

5 Discussion

This study analysed the association between the emotional states “boredom” and “enjoyment” and the severity of the experience of barriers in Massive Open Online Courses. Based on earlier research we expected that boredom is positively associated with the experience of barriers while enjoyment is negatively associated with the experience of barriers. More specifically we expected a negative association between enjoyment and motivation-related barriers like ‘procrastination’, ‘lack of motivation’ and ‘responsibility for one’s own learning’ while assuming a positive association with boredom and these barriers.

Our findings confirm a small statistically significant correlation between boredom and the barrier components ‘Tech and online learning related skills’, ‘social context’ and ‘time, support and motivation’. A lack of skills related to handling the online learning environment can be without question a source of boredom during the activity of learning. By facing this type of barriers the emotional state could potentially decrease the likeliness of overcoming these barriers and finally lead to a stop of learning activities. For the social context component, boredom can arise depending on the expectation of the learners. Some learners might enter the MOOC with the expectation to find an open course with a vibrant learning community while being confronted with mainly content-related interactions that do not require a lot of social activity. Last but not least the correlation to the barrier component ‘time, support and motivation’ can be explained from the perspective of self-regulated learning. Research conducted by Pekrun and others [8] has confirmed that boredom relates negatively to self-regulated learning. Surprisingly, the barrier component ‘course design’ was not significantly associated with boredom. An explanation for this may be that we did not collect data in one specific MOOC, but rather targeted MOOC-learners in general who at some point in the near past participated in a MOOC and were asked to refer to that MOOC when answering the survey questions. As emotions are known to be a reaction to the environment [24], MOOC-specific future research should aim to analyze the association between course designs of MOOCs or specific learning activities embedded into a MOOC environment and achievement emotions.

Our expectations regarding enjoyment and the encounter of barriers could not be confirmed. No significant correlation between any barrier component and the emotional state of enjoyment could be identified. Only when analyzing the motivation-related barriers more fine-grained, we could see that learners with higher levels of enjoyment experience the barrier ‘own responsibility for learning’ as less severe. It is a question for future research how the process of resolving the encountered barriers is influenced by different emotional states. In addition, research by Artino and Jones [27] has shown the complexity and association between self-regulated learning behavior, positive emotional states and metacognitive activities. Future research on barriers in MOOCs needs to further untangle the different background variables of learners, their emotional states during learning, the encounter and solving of different barriers and last but not least the influence of this whole system on individual achievement.

The study has resulted in some interesting findings related to gender differences between emotional states and barriers. Men with higher levels of boredom experience barriers related to ‘technical and online learning related skills’, ‘social context’ and ‘time, support and motivation’ more severe while men with higher levels of enjoyment experience the barriers related to ‘time, support and motivation’ less severe. Female learners who are more bored, on the other hand, experience barriers related to ‘social context’ and ‘time, support and motivation’ more severe. Although the difference between men and women was not found statistically significant, the results provide interesting starting points for further research into gender, emotional states and the experience of barriers. Similar results could be confirmed for motivation related barriers and gender differences.

6 Conclusions

Since most correlations were statistically significant but small, it is an open question how influential emotions are for the experience of barriers. Current findings can be interpreted in two directions: On the one hand, emotions can contribute to the experience of barriers, on the other hand, barriers can also be a source of these emotions. Future research needs to differentiate between these different types of emotions and the direction of their relation to the experienced barriers. Furthermore, future studies should take into consideration that additional variables may influence this relation and the direction of the relationship.

With a focus on enjoyment and boredom, this study has compared an extreme pair of learning-related emotions. Research by D’Mello, Blair, Lehman and Person [34] on affective states during problem solving has shown the importance of analysing the fluctuations between emotional states. The authors recommend to go beyond a basic valence-arousal framework. This recommendation is highly related to the research study at hand. In the current study, we have only focused on a limited set of emotional states. Future work should untangle the flow of emotions during the occurrence of barriers and examine potential problem-solving approaches related to these barriers.

This research has contributed to the emerging field of the role of emotions in open online learning environments. The novelty of the study comes from the theoretical approach used in the study and the new research direction of investigating the connection between emotions and barriers. It would be especially promising if future research were to expand the research focus not only into different types of barriers, but also towards the intrapersonal process that allows learners to cope with the experience of barriers and the control of connected emotions.