Advertisement

Enjoyed or Bored? A Study into Achievement Emotions and the Association with Barriers to Learning in MOOCs

  • Maartje HenderikxEmail author
  • Ansje Lohr
  • Marco Kalz
Open Access
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11722)

Abstract

MOOCs are accessible online personal development opportunities in which learners can expand their knowledge on many topics. Yet, the experience of barriers to learning often hinders learners from achieving their personal learning goals. Therefore, it is important to have insight into determinants that may influence the experience of (certain) barriers. This study investigated whether the emotional determinants enjoyment and boredom, which are known to impact learner achievement and motivation, affect the experience of (specific) barriers while learning in MOOCs. The results show that boredom did affect the experience of barriers related to technical and online related skills, social context and time, support and motivation, yet it did not affect the experience of barriers related to the design of the MOOC. Enjoyment was not correlated to any of the barriers. Furthermore, the same analysis comparing men to women again revealed that boredom did not significantly affect the experience of barriers related to the design of the MOOC, yet did significantly affect the experience of the other barriers. No, significant differences were found between males and females These findings may serve as input for supporting learners in achieving their individual learning goals.

Keywords

MOOCs Barriers Achievement emotions Online learning Open education 

1 Introduction

MOOCs offer learners easy accessible personal development opportunities online, in which they can expand their knowledge on many topics at their own time and pace [1]. These learners each have their individual goals they intent to achieve, which can range from finishing a certain number of modules, following the whole course without doing assignments to completing the course and getting a certificate [2]. However, Henderikx, Kreijns and Kalz [3] found that learners do not always succeed in reaching their individual goals because they encounter barriers to learning. These barriers can be either directly related to the MOOC (e.g. lack of instructor presence, bad course content, lack of instant feedback) or not directly related to the MOOC (e.g. lack of motivation, family issues, technical problems with the pc or internet) and hinder or prevent learners from pursuing their individual learning goals [3, 4]. As barriers seem to have a substantial impact on academic achievement [5, 6], knowledge about factors that affect the encounter of these barriers would be valuable. A recent study by Henderikx, Kreijns and Kalz [7] examined possible factors affecting the encounter of barriers to learning in MOOCs. They found that age (specifically the group between 20–50 years), gender, educational level and previous online learning experience increased the encounter of (certain) barriers in MOOCs.

These factors like age, gender, educational level and previous online learning experience, which have been often studied in various learning contexts (traditional face-2-face contexts, online contexts) specifically in relation to academic achievement, could be classified as “hard” factors as they are visible or quantifiable. “Soft” factors however, which are less tangible and harder to quantify, like achievement emotions also play a part in the encounter of barriers to learning in MOOCs.

Achievement emotions, which can be defined as “emotions tied directly to achievement activities or achievement outcomes” [8], have been identified as very important as they can affect learner engagement, motivation and subsequently academic achievement [9, 10, 11]. Especially the emotions ‘enjoyment’ and ‘boredom’, which are related to the activity of learning, can be activating (enjoyment) or deactivating (boredom) emotions with regard to learning motivation [11]. Most studies, regarding achievement emotions, were set in traditional face to face settings, but an increasing number of studies investigated the influence of learner emotions in online learning environments [12, 13, 14]. However, research using Pekrun et al.’s [10] enjoyment and boredom scales in MOOC settings is sparse. With this paper, we provide two contributions to the field of learning in open online environments: (1) We examine the enjoyment and boredom scales in a MOOC context and (2) we investigate whether there is an association between enjoyment or boredom and the extent to which learners experience barriers in MOOCs.

2 Theoretical Background and Related Work

2.1 Barriers to Learning in MOOCs

Learning and succeeding in open non-formal learning environments like MOOCs can be challenging [15]. A reason why learners do not always reach their personal learning goals is that they encounter barriers to learning, which prevent or hinder them in their learning progress. These barriers can be either MOOC-related or non-MOOC related [3, 4]. Typical MOOC-related barriers often mentioned by learners are lack of interaction [5, 16, 17], lack of instructor presence [17] and course content [5, 18]. Non-MOOC related barriers experienced by learners are insufficient academic knowledge [6, 19, 20], lack of time [5, 6, 19, 21]. A principal component analysis study by Henderikx et al. [4], distinguished four different barrier components and interestingly found that most barriers could be classified as non-MOOC related (see Table 1).
Table 1.

Classification of barrier components [4]

Component

Type

Barrier examples

Coping level

1-Technical and online related skills

Non-MOOC related

• Lack of software skills

• Lack of typing skills

Can be dealt with on a personal level

2-Social context

Partly MOOC and non-MOOC related

• Feeling of isolation

• Lack of student collaboration

Can be dealt with on both personal and MOOC-level

3-Course design

MOOC-related

• Low quality materials

• Lack of timely feedback

Can be dealt with on MOOC-level

4-Time, support and motivation

Non-MOOC related

• Family issues

• Lack of time

• Lack of motivation

Can be dealt with on personal level

Furthermore, the participants in this study indicated that own responsibility for learning, lack of time, bad course content, lack of motivation, low quality of instruction and/or materials and family issues were most often considered as barriers.

2.2 Achievement Emotions

Achievement emotions or academic emotions are essential to understand as they can affect a learner’s effort, motivation to persist and strategies for learning [11]. According to Pekrun and Linnenbrink-Garcia [11] achievement emotions can be either related to the activity of learning i.e. activity-related emotions such as enjoyment or boredom or related to the outcome i.e. outcome-related emotions such as hope and pride. These emotions can occur in class-related, learning-related and test-related environments and can be divided into positive and negative, activating and deactivating emotions with respect to their impact on student performance and motivation (see Table 2; [22]).
Table 2.

Overview achievement emotions adapted from Pekrun [23]

 

Positive activating

Positive deactivating

Negative activating

Negative deactivating

Activity related

Enjoyment

Relaxation

Anger

Boredom

Outcome related

Hope

Relief

Anxiety

Hopelessness

Pride

Contentment

Shame

Despair

For example, enjoyment is regarded as an activity-related positive activating emotion as it is found to enhance effort and motivation [23]. Boredom, on the other hand, is considered an activity-related negative deactivating emotion while it undermines effort and motivation [23]. For the purpose of this study we solely focus on enjoyment and boredom as emotions during learning. As mentioned in the introduction, the majority of the studies about achievement emotions are set in traditional face to face (offline) contexts.

One of the major differences between offline and online learning environments is the lack of interaction between the learner and the instructor when it comes to identifying and responding to the emotional state of the learner [24]. Emotions are a reaction to the environment and thus in the case of limited interaction with an instructor, these emotions are mainly fuelled by the online learning environment [24]. Dillon et al. [13] explored self-reported emotions learners experience while learning in MOOCs. During the runtime of a MOOC, learners were asked, at seven different occasions, to indicate their feelings about the content (i.e. video’s, assessments) by picking 2 emotions. These emotions were in part selected from Pekrun et al. [10]. The results of their study indicated that emotions are content sensitive, meaning that different content evoked different emotions. Furthermore, positive emotions, more specifically enjoyment, were the most often experienced emotions with regard to the various contents and negative emotions affected dropout and thus learner achievement. A study by Tze, Daniels, Buhr and Le [35] has analysed the connection between achievement in MOOCs and emotions. Results show that learners with low levels of boredom and low levels of guilt are more likely to deal with the course material and the instructional logic of the course design.

A recent study by Beirne, Mac Lochlainn and Mhichíl [25], examined self-reported learner emotions in a MOOC. During the 3-week runtime of the MOOC, learners were prompted 6 times per week at various points in the MOOC to indicate their emotions they experienced at that moment in time. Similar to Dillon et al. [13], they found that positive emotions were predominant during the course and that certain content evoked negative emotions which may affect achievement. In addition, a meta-analysis on achievement emotions in technology based learning environments implies that levels of emotions differ across learning environments but that the effect of emotions generally supported the hypotheses that positive emotions like enjoyment are positively related to achievement and that negative emotions like boredom are negatively related to achievement [26]. Furthermore, their meta-analysis also indicated that there is no exclusive effect between gender and positive or negative emotions. These findings indicate that the interaction with the content in MOOCs as opposed to the interaction with an instructor in offline learning environments, recalls positive as well as negative emotions and affects learner achievement regardless of gender.

Nevertheless, our aim was to explore a possible relationship between emotions (enjoyment and boredom) and barriers experienced by learners in MOOCs, as opposed to individual achievement. As online learning in a MOOC is very different from offline classroom learning, regarding time, pace, location and intellectual support [27] boredom might affect the experience of barriers, as this activity-related deactivating emotion was found to activate avoidance motivation [28] and “…triggers impulses to escape the situation” [23, p. 533]. In other words, learners who experience higher levels of boredom may feel the impulse to escape the situation and may therefore be more susceptible to experiencing barriers to learning. Therefore, we expected that boredom is positively associated with the experience of barriers. Learners who experience more boredom also experience barriers to learning in MOOCs as more severe. For enjoyment we expect to find the opposite direction, learners who experience higher levels of enjoyment will experience barriers while learning in MOOCs as less severe. In addition, while enjoyment and boredom are known to affect learner motivation [10, 11], we again predict a negative association between enjoyment and a positive association between boredom and motivation related barriers i.e. ‘procrastination’, ‘motivation’ and ‘responsibility for one’s own learning’ specifically.

3 Method

3.1 Participants

Sample 1.

The participants were individuals who took part in the MOOC on Marine Litter developed by UN environment and the Open University. The MOOC aimed at stimulating leadership and offered opportunities for actionable and change-oriented learning, related to marine litter [29]. It ran from May until July 2017, covering eight blocks for eight weeks by providing not only in-depth knowledge, but also useful tools and instruments in addressing marine litter problems. In total 39 participants completed the survey (26 women, 13 men, Mage = 42,15 years, age range: 25–68 years).

Sample 2.

The participants were individuals who took part in one or more MOOCs in the Spanish language from different MOOC providers and on different topics in the last 2 years and who indicated that we could contact them for further research, regardless of whether or not they successfully achieved their personal goals in these MOOCs. 1618 Potential respondents received an invitation to participate in the survey of whom 299 actually completed the survey (146 women, 153 men, Mage = 47,02 years, age range: 20–83 years). The samples were added together for the purpose of analysis.

3.2 Materials

Achievement Emotions.

The achievement emotions were measured using the respective scales of the Achievement Emotions Questionnaire developed by Pekrun et al. [10]. Participants were asked to refer to the most recent MOOC they participated in when answering these questions related to enjoyment and boredom. As achievement emotions are context specific, it is important to differentiate between academic settings [10]. In addition, achievement emotions may refer to before, during or after learning or to the outcome [11]. For this reason, we focused on the learning related emotions enjoyment and boredom and more specifically only used the items referring to enjoyment (6 items, e.g. ‘I enjoy acquiring new knowledge’) and boredom (9 items, e.g. ‘The material bores me to death’) during learning (the items referring to before and after learning were thus excluded). The items were slightly adjusted to fit the learning context of MOOCs and scored on a five-point Likert scale, 1 = totally disagree and 5 = totally agree.

These scales have been validated in multiple studies in traditional face to face contexts [10], however as the online learning context fundamentally differs from the traditional learning context, the construct validity of both scales was tested using principal axis factoring with direct oblimin rotation. The factorability of the data was supported by the Kaiser-Meyer-Olkin measure that showed a value of .93 [the recommended minimum value is .6; 30, 31] and the Bartlett’s Test of Sphericity which was statistically significant (p < .05). The number of factors was determined by combining the results of the scree-plot which indicated a break after the 2nd factor, the eigenvalues exceeding 1 and the parallel analysis, which produced 2 random eigenvalues smaller than the first 2 eigenvalues of the principal axis factoring analysis. These two factors explain 79,6% of the variance. Furthermore, all items had primary loadings well exceeding the cut-off point of .4 on one single factor. In addition, the standardized factor loadings were mainly between .8 and 1, which can be regarded as excellent quality loadings [32] and showed a very high internal consistency per factor (see Table 3). These indices all point towards a good construct validity.
Table 3.

Factor analysis and scale reliability of enjoyment and boredom

 

Factor 1

Factor 2

α

Enjoyment

  

.91

I enjoy the challenge of learning in this MOOC

.824

  

I enjoy acquiring new knowledge in this MOOC

.813

  

I enjoy dealing with the course material of this MOOC

.881

  

I study more than required in this MOOC because I enjoy it so much

.753

  

When studying in this MOOC is going well, it gives me a rush

.818

  

I get physically excited when studying this MOOC is going well

.635

  

Boredom

  

.98

The MOOC material bores me to death

 

.884

 

Studying this MOOC bores me

 

.936

 

Studying this MOOC is dull and monotonous

 

.937

 

While studying this boring MOOC material, I spend my time thinking of how time stands still

 

.958

 

The MOOC material is so boring that I find myself daydreaming

 

.965

 

I find my mind wandering while studying this MOOC

 

.927

 

Because I’m bored I get tired sitting at my desk

 

.902

 

The MOOC material bores me so much that I feel depleted

 

.990

 

While studying in this MOOC, I seem to drift off because it’s so boring

 

.997

 

Barriers.

A ‘Barriers to MOOC-learning’ survey was developed, which contained items drawn from general online learning, distance education and MOOC-specific context literature on barriers and enablers to learning, as discussed in previous section. After answering several general questions about gender, age, educational background, employment status, MOOC-learning experience and preferred learning context, respondents were asked to indicate to what extent they considered the 44 listed items as barriers to learning in a MOOC on a 5-point Likert scale ranging from 1 = to a very large extent’ to ‘not at all’. Examples of items are ‘lack of decent feedback’, ‘family issues’, ‘technical problems with the computer’ and ‘lack of instructor presence’.

3.3 Procedures

Sample 1.

In week 5 of the MOOC on Marine Litter, participants were invited to complete a survey, via a link in the MOOC, about barriers to learning in the MOOC which also included questions about the experience of enjoyment or boredom during learning. Participation was voluntary and filling out the questionnaire took 5–10 min.

Sample 2.

Over the course of several weeks potential respondents were invited via email batches using the open source online survey tool Limesurvey (visit http://www.limesurvey.org) to complete a survey about barriers to learning in MOOCs which also included questions about the experience of enjoyment or boredom during learning. Participation was voluntary and filling out the questionnaire took 5–10 min. After four and six weeks, a reminder was sent to those who did not yet completed the survey.

3.4 Data Screening

The Mahalanobis distance was calculated to identify possible outliers. Based on these calculations, 34 outliers were determined and removed. Due to the high number of outliers we ran the analyses twice, with and without outliers, to verify whether it would influence the analyses. Yet no difference in outcomes was detected. The final sample of included 304 cases, which exceeds the generally accepted item ratio to conduct a factor analysis of 5 to10 respondents per item [33].

4 Results

The relationships between the four different barrier components as determined by Henderikx et al. [4], the motivation specific barriers and the achievement emotions enjoyment and boredom [10] were investigated using Pearson product-moment correlation coefficient. Table 4 shows the associations between the 4 barrier components and enjoyment and boredom. A small statistically significant correlation was found between boredom and the barrier components 1, 2 and 4, indicating that learners who experience higher levels of boredom experience barriers related to ‘technical and online learning related skills’, ‘social context’ and ‘time, support and motivation’ more severe. Enjoyment was not statistically significantly correlated to any of the barrier components.
Table 4.

Correlations between achievement emotions and barrier components (N = 304)

 

Component 1 tech and online learning related skills

Component 2 social context

Component 3 course design

Component 4 time, support and motivation

Boredom

−.152**

−.192**

−.111

−.254**

Enjoyment

.054

.100

.033

.083

Note: **p <. 01

Table 5 displays the correlation results of the analysis between the motivation related barriers ‘procrastination’, ‘motivation’ and ‘responsibility for one’s own learning’ and enjoyment and boredom. The results indicate a small statistically significant correlation between boredom and each of the motivation related barriers and a small significant correlation between enjoyment and the barrier ‘own responsibility for learning’. Learners with higher levels of boredom experience each of the motivation related barriers more severe and learners with higher levels of enjoyment experience the barrier ‘own responsibility for learning’ less severe.
Table 5.

Correlations between achievement emotions and motivation related barriers (N = 304)

 

Procrastinate

Lack of motivation

Own responsibility for learning

Boredom

−.205**

−.178**

−.201**

Enjoyment

.090

.108

.123*

Note: *p <. 05, **p <. 01

While the findings of Loderer, Pekrun and Lester [26] did not find an exclusive effect between gender and positive or negative emotions, we were interested in whether a difference in association between the variables could be detected for male and female learners. Table 6 shows that, for males, a small statistically significant correlation was found between boredom and the barrier components 1, 2 and 4 and a small statistically significant correlation between enjoyment and the barrier component 4. Men who experience higher levels of boredom experience barriers related to ‘technical and online learning related skills’, ‘social context’ and ‘time, support and motivation’ more severe. In addition, men with higher levels of enjoyment experience the barriers related to ‘time, support and motivation’ less severe. For females, a small statistically significant correlation was detected between boredom and the barrier components 2 and 4, indicating that females who are more bored experience barriers related to ‘social context’ and ‘time, support and motivation’ more severe. No statistically significant correlation was found between female enjoyment and any of the barrier components.
Table 6.

Correlations between achievement emotions and barrier components by gender

 

Component 1 tech and online learning related skills

Component 2 social context

Component 3 Course design

Component 4 time, support and motivation

Male (N = 147)

Boredom

−.181*

−.217**

−.074

−.286**

Enjoyment

.046

.084

.032

.168*

Female (N = 157)

Boredom

−.121

−.175*

−.142

−.220**

Enjoyment

.056

.115

.024

−.001

Note: *p <. 05, **p <. 01

The results in Table 6 showed that for both men and women a small statistically significant correlation was found between boredom and the barrier components 2 and 4. To test whether the difference between these correlations is statistically significant the observed value of z (zobs) was calculated. The zobs for the established correlations between boredom and barrier component 2 and 4 are respectively 0.362313 and 0.646987. These values are both within the range of −1.96 < Zobs < 1.96, which indicates that there is no statistical difference in the strength of the correlation between boredom and barrier components 2 and 4 for males and females.

Furthermore, examining the results of the analysis by gender of the motivation related barriers and enjoyment and boredom in Table 7, it can be inferred that for both males and females a small statistically significant correlation between boredom and each of the motivation related barriers was found. Men as well as women with higher levels of boredom experience each of the motivation related barriers more severe. For neither men nor women an association between enjoyment and the motivation related barriers was found.
Table 7.

Correlations between achievement emotions and motivation related barriers by gender

 

Procrastinate

Lack of motivation

Own responsibility for learning

Male (N = 147)

Boredom

−.246**

−.164*

−.230**

Enjoyment

.042

.097

.151

Female (N = 157)

Boredom

−.180*

−.187*

−.237**

Enjoyment

.138

.112

.101

Note: *p <. 05, **p <. 01

Again, the zobs was calculated to establish whether the difference between the found correlations for men and women was statistically significant. The zobs for the correlations between boredom and the motivation related barriers ‘procrastination’, ‘motivation’ and ‘responsibility for one’s own learning’ were respectively 0.629734, −0.26742 and −0.094891. Similar to previous found scores, these values are all within the range of −1.96 < zobs < 1.96, and thus indicate that there is no statistical difference in the strength of the correlation between boredom and each of the motivation related barriers for males and females.

5 Discussion

This study analysed the association between the emotional states “boredom” and “enjoyment” and the severity of the experience of barriers in Massive Open Online Courses. Based on earlier research we expected that boredom is positively associated with the experience of barriers while enjoyment is negatively associated with the experience of barriers. More specifically we expected a negative association between enjoyment and motivation-related barriers like ‘procrastination’, ‘lack of motivation’ and ‘responsibility for one’s own learning’ while assuming a positive association with boredom and these barriers.

Our findings confirm a small statistically significant correlation between boredom and the barrier components ‘Tech and online learning related skills’, ‘social context’ and ‘time, support and motivation’. A lack of skills related to handling the online learning environment can be without question a source of boredom during the activity of learning. By facing this type of barriers the emotional state could potentially decrease the likeliness of overcoming these barriers and finally lead to a stop of learning activities. For the social context component, boredom can arise depending on the expectation of the learners. Some learners might enter the MOOC with the expectation to find an open course with a vibrant learning community while being confronted with mainly content-related interactions that do not require a lot of social activity. Last but not least the correlation to the barrier component ‘time, support and motivation’ can be explained from the perspective of self-regulated learning. Research conducted by Pekrun and others [8] has confirmed that boredom relates negatively to self-regulated learning. Surprisingly, the barrier component ‘course design’ was not significantly associated with boredom. An explanation for this may be that we did not collect data in one specific MOOC, but rather targeted MOOC-learners in general who at some point in the near past participated in a MOOC and were asked to refer to that MOOC when answering the survey questions. As emotions are known to be a reaction to the environment [24], MOOC-specific future research should aim to analyze the association between course designs of MOOCs or specific learning activities embedded into a MOOC environment and achievement emotions.

Our expectations regarding enjoyment and the encounter of barriers could not be confirmed. No significant correlation between any barrier component and the emotional state of enjoyment could be identified. Only when analyzing the motivation-related barriers more fine-grained, we could see that learners with higher levels of enjoyment experience the barrier ‘own responsibility for learning’ as less severe. It is a question for future research how the process of resolving the encountered barriers is influenced by different emotional states. In addition, research by Artino and Jones [27] has shown the complexity and association between self-regulated learning behavior, positive emotional states and metacognitive activities. Future research on barriers in MOOCs needs to further untangle the different background variables of learners, their emotional states during learning, the encounter and solving of different barriers and last but not least the influence of this whole system on individual achievement.

The study has resulted in some interesting findings related to gender differences between emotional states and barriers. Men with higher levels of boredom experience barriers related to ‘technical and online learning related skills’, ‘social context’ and ‘time, support and motivation’ more severe while men with higher levels of enjoyment experience the barriers related to ‘time, support and motivation’ less severe. Female learners who are more bored, on the other hand, experience barriers related to ‘social context’ and ‘time, support and motivation’ more severe. Although the difference between men and women was not found statistically significant, the results provide interesting starting points for further research into gender, emotional states and the experience of barriers. Similar results could be confirmed for motivation related barriers and gender differences.

6 Conclusions

Since most correlations were statistically significant but small, it is an open question how influential emotions are for the experience of barriers. Current findings can be interpreted in two directions: On the one hand, emotions can contribute to the experience of barriers, on the other hand, barriers can also be a source of these emotions. Future research needs to differentiate between these different types of emotions and the direction of their relation to the experienced barriers. Furthermore, future studies should take into consideration that additional variables may influence this relation and the direction of the relationship.

With a focus on enjoyment and boredom, this study has compared an extreme pair of learning-related emotions. Research by D’Mello, Blair, Lehman and Person [34] on affective states during problem solving has shown the importance of analysing the fluctuations between emotional states. The authors recommend to go beyond a basic valence-arousal framework. This recommendation is highly related to the research study at hand. In the current study, we have only focused on a limited set of emotional states. Future work should untangle the flow of emotions during the occurrence of barriers and examine potential problem-solving approaches related to these barriers.

This research has contributed to the emerging field of the role of emotions in open online learning environments. The novelty of the study comes from the theoretical approach used in the study and the new research direction of investigating the connection between emotions and barriers. It would be especially promising if future research were to expand the research focus not only into different types of barriers, but also towards the intrapersonal process that allows learners to cope with the experience of barriers and the control of connected emotions.

Notes

Acknowledgement

This work is financed via a grant by the Dutch National Initiative for Education Research (NRO)/The Netherlands Organisation for Scientific Research (NWO) and the Dutch Ministry of Education, Culture and Science under the grant nr. 405-15-705 (SOONER/http://sooner.nu).

References

  1. 1.
    Hew, K.F.: Promoting engagement in online courses: what strategies can we learn from three highly rated MOOCS. Br. J. Educ. Technol. 47(2), 320–341 (2016)CrossRefGoogle Scholar
  2. 2.
    Henderikx, M.A., Kreijns, K., Kalz, M.: Refining success and dropout in massive open online courses based on the intention–behavior gap. Distance Educ. 38(3), 353–368 (2017).  https://doi.org/10.1080/01587919.2017.1369006CrossRefGoogle Scholar
  3. 3.
    Henderikx, M., Kreijns, K., Kalz, M.: Intention-behaviour dynamics in MOOCs learning; what happens to good intentions along the way? In: 2018 Learning With MOOCS (LWMOOCS), pp. 110–112. IEEE, September 2018.  https://doi.org/10.1109/lwmoocs.2018.8534595
  4. 4.
    Henderikx, M., Kreijns, K., Kalz, M.: A classification of barriers that influence intention achievement in MOOCs. In: Pammer-Schindler, V., Pérez-Sanagustín, M., Drachsler, H., Elferink, R., Scheffel, M. (eds.) EC-TEL 2018. LNCS, vol. 11082, pp. 3–15. Springer, Cham (2018).  https://doi.org/10.1007/978-3-319-98572-5_1CrossRefGoogle Scholar
  5. 5.
    Hone, K.S., El Said, G.R.: Exploring the factors affecting MOOC retention: a survey study. Comput. Educ. 98, 157–168 (2016).  https://doi.org/10.1016/j.compedu.2016.03.016CrossRefGoogle Scholar
  6. 6.
    Shapiro, H.B., Lee, C.H., Roth, N.E.W., Li, K., Çetinkaya-Rundel, M., Canelas, D.A.: Understanding the massive open online course (MOOC) student experience: an examination of attitudes, motivations, and barriers. Comput. Educ. 110, 35–50 (2017).  https://doi.org/10.1016/j.compedu.2017.03.003CrossRefGoogle Scholar
  7. 7.
    Henderikx, M., Kreijns, K., Castaño Muñoz, J., Kalz, M.: What hinders learners in pursuing goals in MOOCs? An empirical study on factors influencing barriers to learning. Distance Educ. 40(2) (2019).  https://doi.org/10.1080/01587919.2019.1600364CrossRefGoogle Scholar
  8. 8.
    Pekrun, R., Frenzel, A.C., Goetz, T., Perry, R.P.: The control-value theory of achievement emotions: an integrative approach to emotions in education. Bibliothek der Universität Konstanz (2007)CrossRefGoogle Scholar
  9. 9.
    Lüftenegger, M., Klug, J., Harrer, K., Langer, M., Spiel, C., Schober, B.: Students’ achievement goals, learning-related emotions and academic achievement. Front. Psychol. 7 (2016)Google Scholar
  10. 10.
    Pekrun, R., Goetz, T., Frenzel, A.C., Barchfeld, P., Perry, R.P.: Measuring emotions in students’ learning and performance: the achievement emotions questionnaire (AEQ). Contemp. Educ. Psychol. 36(1), 36–48 (2011)CrossRefGoogle Scholar
  11. 11.
    Pekrun, R., Linnenbrink-Garcia, L.: Academic emotions and student engagement. In: Christenson, S., Reschly, A., Wylie, C. (eds.) Handbook of Research on Student Engagement, pp. 259–282. Springer, Boston (2012).  https://doi.org/10.1007/978-1-4614-2018-7_12CrossRefGoogle Scholar
  12. 12.
    Baker, R.S., D’Mello, S.K., Rodrigo, M.M.T., Graesser, A.C.: Better to be frustrated than bored: the incidence, persistence, and impact of learners’ cognitive–affective states during interactions with three different computer-based learning environments. Int. J. Hum Comput Stud. 68(4), 223–241 (2010)CrossRefGoogle Scholar
  13. 13.
    Dillon, J., et al.: Student emotion, co-occurrence, and dropout in a MOOC context. In: EDM, pp. 353–357 (2016)Google Scholar
  14. 14.
    Leony, D., Merino, P.J.M., Valiente, J.A.R., Pardo, A., Kloos, C.D.: Detection and evaluation of emotions in massive open online courses. J. UCS 21(5), 638–655 (2015)Google Scholar
  15. 15.
    Misopoulos, F., Argyropoulou, M., Tzavara, D.: Exploring the factors affecting student academic performance in online programs: a literature review. In: Khare, A., Hurst, D. (eds.) On the Line, pp. 235–250. Springer, Cham (2018).  https://doi.org/10.1007/978-3-319-62776-2_18CrossRefGoogle Scholar
  16. 16.
    Khalil, H., Ebner, M.: Interaction possibilities in MOOCs – how do they actually happen? In: International Conference on Higher Education Development, pp. 1–24. Mansoura University, Egypt (2013)Google Scholar
  17. 17.
    Onah, D.F., Sinclair, J., Boyatt, R.: Dropout rates of massive open online courses: behavioural patterns. In: International Conference on Education and New Learning Technologies, EDULEARN14 Proceedings, Barcelona, vol. 1, pp. 5825–5834 (2014)Google Scholar
  18. 18.
    Adamopoulos, P.: What makes a great MOOC? An interdisciplinary analysis of student retention in online courses. In: Proceedings of the Thirty Fourth International Conference on Information Systems, Milan, Italy (2013)Google Scholar
  19. 19.
    Belanger, Y., Thornton, J.: Bioelectricity: a quantitative approach Duke University’s First MOOC (2013)Google Scholar
  20. 20.
    Kizilcec, R.F., Saltarelli, A.J., Reich, J., Cohen, G.L.: Closing global achievement gaps in MOOCs. Science 355(6322), 251–252 (2017)CrossRefGoogle Scholar
  21. 21.
    Khalil, H., Ebner, M.: MOOCs completion rates and possible methods to improve retention - a literature review. In: World Conference on Educational Multimedia, Hypermedia and Telecommunications, pp. 1236–1244. AACE, Chesapeak (2014)Google Scholar
  22. 22.
    Pekrun, R.: The impact of emotions on learning and achievement: towards a theory of cognitive/motivational mediators. Appl. Psychol. 41(4), 359–376 (1992)CrossRefGoogle Scholar
  23. 23.
    Pekrun, R., Goetz, T., Daniels, L.M., Stupnisky, R.H., Perry, R.P.: Boredom in achievement settings: exploring control–value antecedents and performance outcomes of a neglected emotion. J. Educ. Psychol. 102(3), 531–549 (2010)CrossRefGoogle Scholar
  24. 24.
    Swerdloff, M.: Online learning, multimedia, and emotions. In: Emotions, Technology, and Learning, pp. 155–175 (2016)CrossRefGoogle Scholar
  25. 25.
    Beirne, E., Mac Lochlainn, C., Mhichíl, M.N.G.: Moody MOOCS: an exploration of emotion in an LMOOC. In: Towards Personalized Guidance and Support for Learning, vol. 22 (2018)Google Scholar
  26. 26.
    Loderer, K., Pekrun, R., Lester, J.C.: Beyond cold technology: a systematic review and meta-analysis on emotions in technology-based learning environments. Learning and Instruction (in press)Google Scholar
  27. 27.
    Artino Jr., A.R., Jones II, K.D.: Exploring the complex relations between achievement emotions and self-regulated learning behaviors in online learning. Internet High. Educ. 15(3), 170–175 (2012)CrossRefGoogle Scholar
  28. 28.
    Pekrun, R.: Facets of adolescents’ academic motivation: a longitudinal expectancy-value approach. Adv. Motiv. Achievement 8, 139–189 (1993)Google Scholar
  29. 29.
    Löhr, A.J., Savelli, H., Beunen, R., Kalz, M., Ragas, A., Van Belleghem, F.: Solutions for global marine litter pollution. Curr. Opin. Environ. Sustain. 28, 90–99 (2017)CrossRefGoogle Scholar
  30. 30.
    Kaiser, H.F.: A second-generation little jiffy. Psychometrika 35(4), 401–415 (1970)CrossRefGoogle Scholar
  31. 31.
    Kaiser, H.F.: An index of factorial simplicity. Psychometrika 39(1), 31–36 (1974)CrossRefGoogle Scholar
  32. 32.
    McNeish, D., An, J., Hancock, G.R.: The thorny relation between measurement quality and fit index cutoffs in latent variable models. J. Pers. Assess. 100(1), 43–52 (2018)CrossRefGoogle Scholar
  33. 33.
    Comrey, A.L., Lee, H.B.: A First Course in Factor Analysis, 2nd edn. Lawrence Erlbaum, Hillsdale (1992)Google Scholar
  34. 34.
    D’Mello, S.K., Lehman, B., Person, N.: Monitoring affect states during effortful problem solving activities. Int. J. Artif. Intell. Educ. 20(4), 361–389 (2010)Google Scholar
  35. 35.
    Tze, V., Daniels, L.M., Buhr, E., Le, L.: Affective profiles in a massive open online course and their relationship with engagement. Front. Educ. 2. (2017)Google Scholar

Copyright information

© The Author(s) 2019

Open Access This chapter is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, a link is provided to the Creative Commons license and any changes made are indicated.

The images or other third party material in this chapter are included in the work's Creative Commons license, unless indicated otherwise in the credit line; if such material is not included in the work's Creative Commons license and the respective action is not permitted by statutory regulation, users will need to obtain permission from the license holder to duplicate, adapt or reproduce the material.

Authors and Affiliations

  1. 1.Welten InstituteOpen University of the NetherlandsHeerlenThe Netherlands
  2. 2.Heidelberg University of EducationHeidelbergGermany

Personalised recommendations