Abstract
There are significant complexities in interpreting and measuring quality in MOOCs. In this chapter, we examine experts’ perceptions of how to measure quality in MOOCs, using empirical data we gathered through conversations with MOOC specialists. In their experience, while data can be helpful in understanding quality, the metrics measured are shaped by underpinning assumptions and biases. In conventional education, it is assumed that the learner wants to follow a course pathway and complete a course. However, this assumption may not be valid in a MOOC. Quality data might not capture the underlying goals and intentions of MOOC learners. Therefore, it is difficult to measure whether or not a learner has achieved his or her goals. We stress the need to explore quality metrics from the learner’s point of view and to encompass the variability in motivations, needs and backgrounds, which shape conceptions of quality for individuals.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Campbell’s law—The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.
References
Adamopoulos, A. (2013). What makes a great MOOC? An interdisciplinary analysis of student retention in online courses. Paper presented at the Thirty-Fourth International Conference on Information Systems, Milan, Italy. Retrieved from http://pages.stern.nyu.edu/~padamopo/What%20makes%20a%20great%20MOOC.pdf.
Alario-Hoyos, C., Perez-Sanagustin, M., Cormier, D., & Delgado-Kloos, C. (2014). Proposal for a conceptual framework for educators to describe and design MOOCs. Journal of Universal Computer Science, 20(1), 6–23.
Amo, D. (2013, November). MOOCs: Experimental approaches for quality in pedagogical and design fundamentals. Paper presented at TEEM’13, Salamanca, Spain.
Bayne, S., & Ross, J. (2014). MOOC pedagogy. In P. Kim (Ed.), Massive open online courses: The MOOC revolution (pp. 23–45). New York, NY: Routledge.
Biesta, G. (2007). Why “what works” won’t work: Evidence-based practice and the demcrative deficit in educational research. Educational Theory, 57(1), 1–22.
Biggs, J. (1993). From theory to practice: A cognitive systems approach. Higher Education Research & Development, 12(1), 73–85.
Chandrasekaran, M., Ragupathi, K., Kan, M., & Tan, B. (2015, December). Towards feasible instructor intervention in MOOC discussion forums. Paper presented at the Thirty-Sixth International Conference on Information Systems, Fort Worth, TX.
Coetzee, D., Lim, S., Fox, A., Hartmann, B., & Hearst, M. A. (2015). Structuring interactions for large-scale synchronous peer learning. In Proceedings of the 18th ACM Conference on Computer- Supported Cooperative Work and Social Computing (CSCW), Vancouver, Canada (pp. 1139–1152). New York, NY: ACM.
Conole, G. (2013). MOOCs as disruptive technologies: Strategies for enhancing the learner experience and quality of MOOCs. RED—Revista de Educación a Distancia, 39. Retrieved from http://www.um.es/ead/red/39/conole.pdf.
Daradoumis, T., Bassi, R., Xhafa, F., & Caballé, S. (2013, October). A review on massive e-learning (MOOC) design, delivery and assessment. In 2013 Eighth International Conference on P2P, Parallel, Grid, Cloud and Internet Computing (3PGCIC) (pp. 208–213). Piscataway, NJ: IEEE.
DeBoer, J., Ho, A., Stump, G. S., & Breslow, L. (2014). Changing “course”: Reconceptualizing educational variables for massive open online courses. Educational Researcher, 43(2), 74–84. https://doi.org/10.3102/0013189X14523038.
Deslauriers, L., Schelew, E., & Wieman, C. (2011). Improved learning in a large-enrolment physics class. Science, 332(6031), 862–864.
Dillenbourg, P., Fox, A., Kirchner, C., Mitchell, J., & Wirsing, M. (2014). Massive open online courses: Current state and perspectives. Manifesto from Dagstuhl Perspectives Workshop. https://doi.org/10.4230/DagMan.4.1.1.
Downes, S. (2013, April 24). The quality of massive open online courses. Retrieved from http://mooc.efquel.org/files/2013/05/week2-The-quality-of-massive-open-online-courses-StephenDownes.pdf.
Fenwick, T. (2015). Sociomateriality and learning: A critical approach. In D. Scott & E. Hargreaves (Eds.), The SAGE handbook of learning (online). London: SAGE.
Gibbs, G. (2010). Dimensions of quality. York, UK: The Higher Education Academy.
Gillani, N., & Eynon, R. (2014). Communication patterns in massively open online courses. Internet and Higher Education, 23, 18–26.
Grover, S., Franz, P., Schneider, E., & Pea, R. (2013). The MOOC as distributed intelligence: Dimension of a framework an evaluation of MOOCs. Paper presented at the 10th Annual International Conference on Computer Supported Collaborative Learning, Madison, WI. Retrieved from http://life-slc.org/docs/LSLC_rp_A194_Grover-etal_CSCL2013_MOOCsand-DI_Volume%202_CSCL2013.pdf.
Grunewald, F., Meinel, C., Totschnig, M., & Willems, C. (2013). Designing MOOCs for the support of multiple learning styles. In Conference Proceedings from EC-TEL 2013, LNCS (pp. 371–382). Berlin, Germany: Springer.
Guardia, L., Maina, M., & Sangra, A. (2013). MOOC design principles: A pedagogical approach from the learner’s perspective. eLearning Papers, 33, 1–5.
Guo, P., Kim, J., & Rubin, R. (2014). How video production affects student engagement: An empirical study of MOOC videos. In Proceedings of the First ACM conference on Learning @ Scale Conference (pp. 41–50). New York, NY: ACM.
Hew, K. (2014). Promoting engagement in online courses: What strategies can we learn from three highly rated MOOCS? British Journal of Educational Technology, 47(2), 320–342. https://doi.org/10.1111/bjet.12235.
Hood, N., & Littlejohn, A. (2016a). MOOC Quality: A call for new quality measures. Journal of Learning for Development, 3(3), 28–42 https://t.co/EePAUPWnDb.
Hood, N., & Littlejohn, A. (2016b). Quality in MOOCs, surveying the terrain (Commonwealth for Learning Report). http://oasis.col.org/handle/11599/2352.
Howley, I., Mayfield, E., & Rosé, C. P. (2013). Linguistic analysis methods for studying small groups. In C. Hmelo-Silver, A. O’Donnell, C. Chan, & C. Chin (Eds.), The international handbook of collaborative learning (pp. 184–202). New York, NY: Routledge.
Illeris, K. (2007). How we learn: Learning and non-learning in school and beyond. London: Routledge.
Istrate, O., & Kestens, A. (2015, April). Developing and monitoring a MOOC: The IFRC experience. Paper presented at the 11th International Scientific Conference eLearning and Software for Education, Bucharest, Romania. Retrieved from http://www.academia.edu/14707457/DEVELOPING_AND_MONITORING_A_MOOC_THE_IFRC_EXPERIENCE.
Jordan, K. (2015). Massive open online course completion rates revisited: Assessment, length and attrition. International Review of Research in Open and Distributed Learning, 16(3), 341–358.
Kanwar, A. (2013, October 16). Quality vs. quantity: Can technology help? Opening keynote presentation at the 25th ICDE World Conference, Tianjin, China.
Kay, J., Reimann, P., Diebold, E., & Kummerfeld, B. (2013). MOOCs: So many learners, so much potential …. IEEE Intelligent Systems, 28(3), 70–77 (Kizilcic et al., 2013).
Kizilcec, R., Piech, C., & Schneider, E. (2013). Deconstructing disengagement: Analyzing learner subpopulations in massive open online courses. LAK’13 Leuven, Belgium.
Kling, R., & Courtright, C. (2003). Group behavior and learning in electronic forums: A sociotechnical approach. The Information Society, 19, 221–235.
Kolowich, S. (2013, March 18). The professors behind the MOOC hype. The Chronicle of Higher Education. Retrieved from http://chronicle.com/article/The-Professors-Behind-the-MOOC/137905/.
Lackner, E., Ebner, M., & Khalil, M. (2015). MOOCs as granular systems: Design patterns to foster participant activity. eLearning Papers, 42, 28–37.
Lin, Y.-L., Lin, H.-W., & Hung, T.-T. (2015). Value hierarchy for massive open online courses. Computers in Human Behaviour, 53, 408–418.
Littlejohn, A., Hood, N., Milligan, C., & Mustain, P. (2016). Learning in MOOCs: Motivations and self-regulated learning in MOOCs. The Internet and Higher Education, 29, 40–48. https://doi.org/10.1016/j.iheduc.2015.12.003.
Mak, S., Williams, R., & Mackness, J. (2010). Blogs and forums as communication and learning tools in a MOOC. In L. Dirckinck-Holmfeld, V. Hodgson, C. Jones, M. de Laat, D. McConnell, & T. Ryberg (Eds.), In Proceedings of the 7th International Conference on Networked Learning 2010 (pp. 275–284). Lancaster, UK: Lancaster University. Retrieved from https:// www.lancaster.ac.uk/fss/organisations/netlc/past/nlc2010/abstracts/PDFs/Mak.pdf.
Mamgain, N., Sharma, A., & Goyal, P. (2014). Learner’s perspective on video-viewing features offered by MOOC providers: Coursera and edX. Paper presented at the 2014 IEEE International Conference on MOOC, Innovation and Technology in Education (MITE). https://doi.org/10.1109/mite.2014.7020298.
Margaryan, A., Bianco, M., & Littlejohn, A. (2015). Instructional quality of massive open online courses (MOOCs). Computers & Education, 80, 77–83.
Morozov, E. (2014). The planning machine. The New Yorker, 13 October (www.newyorker.com/magazine/2014/10/13/planning-machine.
Morrison, D. (2014, January 18). Need-to-know-news: MOOC mentors for hire, Coursera’s MOC$s, edX shares MOOC data and more. Online Learning Insights. Retrieved from https://onlinelearninginsights.wordpress.com/2014/01/28/need-to-know-news-mooc-mentors-forhire-courseras-mocs-edx-shares-mooc-data-and-more/.
Perna, L., Ruby, A., Boruch, R., Wang, N., Scull, J., Ahmad, S., et al. (2014). Moving through MOOCs: Understanding the progression of users in massive open online courses. Education Researcher, 43(9), 421–432.
Rodriguez, C. (2012). MOOCs and the AI-Stanford like courses: Two successful and distinct course formats for massive open online courses. European Journal of Open, Distance and E-Learning. Retrieved from http://files.eric.ed.gov/fulltext/EJ982976.pdf.
Ross, J., Sinclair, C., Knox, J., & Macleod, H. (2014). Teacher experiences and academic identity: The missing components of MOOC pedagogy. Journal of Online Learning and Teaching, 10(1), 57.
Scagnoli, N. (2012). Thoughts on instructional design for MOOCs. Retrieved from https://www.ideals.illinois.edu/bitstream/handle/2142/44835/Instructional%20Design%20of%20a%20MOOC.pdf.
Selwyn, N. (2014). Distrusting educational technology. London and New York: Routledge.
Selwyn, N. (2016). Is technology good for education. Cambridge, UK: Polity Books.
Shah, D. (2016). By the numbers: MOOCS in 2016. [Online]. Retrieved from: https://www.class-central.com/report/mooc-stats-2016/.
Sharples, M., & Domingue, J. (2016, September). The blockchain and kudos: A distributed system for educational record, reputation and reward. In European Conference on Technology Enhanced Learning (pp. 490–496). Springer International Publishing.
Sinha, T., Li, N., Jermann, P., & Dillenbourg, P. (2014). Capturing “attrition intensifying” structural traits from didactic interaction sequences of MOOC learners. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Workshop on Modeling Large Scale Social Interaction in Massively Open Online Courses (pp. 42–49). Retrieved from https://www.aclweb.org/anthology/W/W14/W14-41.pdf.
Tabba, Y., & Medouri, A. (2013). LASyM: A learning analytics system for MOOCs. International Journal of Advanced Computer Science and Applications, 4(5), 113–119.
Tyler, K. (1939). Recent developments in radio education. The English Journal, 28(3), 193–199.
Warburton, S., & Mor, Y. (2015). Double loop design: Configuring narratives, patterns and scenarios in the design of technology enhanced learning. In M. Maina et al. (Eds.), The art and science of learning design (pp. 93–104). Rotterdam, The Netherlands: Sense Publishers.
Wen, M., Yang, D., & Rosé, C. P. (2014a). Linguistic reflections of student engagement in massive open online courses. In Proceedings of the International Conference on Weblogs and Social Media. Retrieved from http://www.cs.cmu.edu/~mwen/papers/icwsm2014-camera-ready.pdf.
Wen, M., Yang, D., & Rosé, C. P. (2014b). Sentiment analysis in MOOC discussion forums: What does it tell us? In Proceedings of Educational Data Mining. Retrieved from http://www.cs.cmu.edu/~mwen/papers/edm2014-camera-ready.pdf.
Yang, D., Wen, M., Kumar, A., Xing, E., & Rosé, C. (2014). Towards an integration of text and graph clustering methods as a lens for studying social interaction in MOOCs. International Review of Research in Open and Distributed Learning, 15(5). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1853/3083.
Acknowledgements
The authors wish to thank Vasudha Chaudhari of The Open University for comments and for proofing this chapter.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2018 The Author(s)
About this chapter
Cite this chapter
Littlejohn, A., Hood, N. (2018). Designing for Quality?. In: Reconceptualising Learning in the Digital Age. SpringerBriefs in Education(). Springer, Singapore. https://doi.org/10.1007/978-981-10-8893-3_5
Download citation
DOI: https://doi.org/10.1007/978-981-10-8893-3_5
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-8892-6
Online ISBN: 978-981-10-8893-3
eBook Packages: EducationEducation (R0)