Skip to main content
SpringerLink
Log in

Designing for Quality?

  • Chapter
  • First Online:
Book cover Reconceptualising Learning in the Digital Age

Part of the book series: SpringerBriefs in Education ((BRIEFSODE))

  • 540 Accesses

Abstract

There are significant complexities in interpreting and measuring quality in MOOCs. In this chapter, we examine experts’ perceptions of how to measure quality in MOOCs, using empirical data we gathered through conversations with MOOC specialists. In their experience, while data can be helpful in understanding quality, the metrics measured are shaped by underpinning assumptions and biases. In conventional education, it is assumed that the learner wants to follow a course pathway and complete a course. However, this assumption may not be valid in a MOOC. Quality data might not capture the underlying goals and intentions of MOOC learners. Therefore, it is difficult to measure whether or not a learner has achieved his or her goals. We stress the need to explore quality metrics from the learner’s point of view and to encompass the variability in motivations, needs and backgrounds, which shape conceptions of quality for individuals.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Campbell’s law—The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.

References

  • Adamopoulos, A. (2013). What makes a great MOOC? An interdisciplinary analysis of student retention in online courses. Paper presented at the Thirty-Fourth International Conference on Information Systems, Milan, Italy. Retrieved from http://pages.stern.nyu.edu/~padamopo/What%20makes%20a%20great%20MOOC.pdf.

  • Alario-Hoyos, C., Perez-Sanagustin, M., Cormier, D., & Delgado-Kloos, C. (2014). Proposal for a conceptual framework for educators to describe and design MOOCs. Journal of Universal Computer Science, 20(1), 6–23.

    Google Scholar 

  • Amo, D. (2013, November). MOOCs: Experimental approaches for quality in pedagogical and design fundamentals. Paper presented at TEEM’13, Salamanca, Spain.

    Google Scholar 

  • Bayne, S., & Ross, J. (2014). MOOC pedagogy. In P. Kim (Ed.), Massive open online courses: The MOOC revolution (pp. 23–45). New York, NY: Routledge.

    Google Scholar 

  • Biesta, G. (2007). Why “what works” won’t work: Evidence-based practice and the demcrative deficit in educational research. Educational Theory, 57(1), 1–22.

    Article  Google Scholar 

  • Biggs, J. (1993). From theory to practice: A cognitive systems approach. Higher Education Research & Development, 12(1), 73–85.

    Article  Google Scholar 

  • Chandrasekaran, M., Ragupathi, K., Kan, M., & Tan, B. (2015, December). Towards feasible instructor intervention in MOOC discussion forums. Paper presented at the Thirty-Sixth International Conference on Information Systems, Fort Worth, TX.

    Google Scholar 

  • Coetzee, D., Lim, S., Fox, A., Hartmann, B., & Hearst, M. A. (2015). Structuring interactions for large-scale synchronous peer learning. In Proceedings of the 18th ACM Conference on Computer- Supported Cooperative Work and Social Computing (CSCW), Vancouver, Canada (pp. 1139–1152). New York, NY: ACM.

    Google Scholar 

  • Conole, G. (2013). MOOCs as disruptive technologies: Strategies for enhancing the learner experience and quality of MOOCs. RED—Revista de Educación a Distancia, 39. Retrieved from http://www.um.es/ead/red/39/conole.pdf.

  • Daradoumis, T., Bassi, R., Xhafa, F., & Caballé, S. (2013, October). A review on massive e-learning (MOOC) design, delivery and assessment. In 2013 Eighth International Conference on P2P, Parallel, Grid, Cloud and Internet Computing (3PGCIC) (pp. 208–213). Piscataway, NJ: IEEE.

    Google Scholar 

  • DeBoer, J., Ho, A., Stump, G. S., & Breslow, L. (2014). Changing “course”: Reconceptualizing educational variables for massive open online courses. Educational Researcher, 43(2), 74–84. https://doi.org/10.3102/0013189X14523038.

    Article  Google Scholar 

  • Deslauriers, L., Schelew, E., & Wieman, C. (2011). Improved learning in a large-enrolment physics class. Science, 332(6031), 862–864.

    Article  Google Scholar 

  • Dillenbourg, P., Fox, A., Kirchner, C., Mitchell, J., & Wirsing, M. (2014). Massive open online courses: Current state and perspectives. Manifesto from Dagstuhl Perspectives Workshop. https://doi.org/10.4230/DagMan.4.1.1.

    Article  Google Scholar 

  • Downes, S. (2013, April 24). The quality of massive open online courses. Retrieved from http://mooc.efquel.org/files/2013/05/week2-The-quality-of-massive-open-online-courses-StephenDownes.pdf.

  • Fenwick, T. (2015). Sociomateriality and learning: A critical approach. In D. Scott & E. Hargreaves (Eds.), The SAGE handbook of learning (online). London: SAGE.

    Google Scholar 

  • Gibbs, G. (2010). Dimensions of quality. York, UK: The Higher Education Academy.

    Google Scholar 

  • Gillani, N., & Eynon, R. (2014). Communication patterns in massively open online courses. Internet and Higher Education, 23, 18–26.

    Article  Google Scholar 

  • Grover, S., Franz, P., Schneider, E., & Pea, R. (2013). The MOOC as distributed intelligence: Dimension of a framework an evaluation of MOOCs. Paper presented at the 10th Annual International Conference on Computer Supported Collaborative Learning, Madison, WI. Retrieved from http://life-slc.org/docs/LSLC_rp_A194_Grover-etal_CSCL2013_MOOCsand-DI_Volume%202_CSCL2013.pdf.

  • Grunewald, F., Meinel, C., Totschnig, M., & Willems, C. (2013). Designing MOOCs for the support of multiple learning styles. In Conference Proceedings from EC-TEL 2013, LNCS (pp. 371–382). Berlin, Germany: Springer.

    Google Scholar 

  • Guardia, L., Maina, M., & Sangra, A. (2013). MOOC design principles: A pedagogical approach from the learner’s perspective. eLearning Papers, 33, 1–5.

    Google Scholar 

  • Guo, P., Kim, J., & Rubin, R. (2014). How video production affects student engagement: An empirical study of MOOC videos. In Proceedings of the First ACM conference on Learning @ Scale Conference (pp. 41–50). New York, NY: ACM.

    Google Scholar 

  • Hew, K. (2014). Promoting engagement in online courses: What strategies can we learn from three highly rated MOOCS? British Journal of Educational Technology, 47(2), 320–342. https://doi.org/10.1111/bjet.12235.

    Article  Google Scholar 

  • Hood, N., & Littlejohn, A. (2016a). MOOC Quality: A call for new quality measures. Journal of Learning for Development, 3(3), 28–42 https://t.co/EePAUPWnDb.

  • Hood, N., & Littlejohn, A. (2016b). Quality in MOOCs, surveying the terrain (Commonwealth for Learning Report). http://oasis.col.org/handle/11599/2352.

  • Howley, I., Mayfield, E., & Rosé, C. P. (2013). Linguistic analysis methods for studying small groups. In C. Hmelo-Silver, A. O’Donnell, C. Chan, & C. Chin (Eds.), The international handbook of collaborative learning (pp. 184–202). New York, NY: Routledge.

    Google Scholar 

  • Illeris, K. (2007). How we learn: Learning and non-learning in school and beyond. London: Routledge.

    Google Scholar 

  • Istrate, O., & Kestens, A. (2015, April). Developing and monitoring a MOOC: The IFRC experience. Paper presented at the 11th International Scientific Conference eLearning and Software for Education, Bucharest, Romania. Retrieved from http://www.academia.edu/14707457/DEVELOPING_AND_MONITORING_A_MOOC_THE_IFRC_EXPERIENCE.

  • Jordan, K. (2015). Massive open online course completion rates revisited: Assessment, length and attrition. International Review of Research in Open and Distributed Learning, 16(3), 341–358.

    Article  Google Scholar 

  • Kanwar, A. (2013, October 16). Quality vs. quantity: Can technology help? Opening keynote presentation at the 25th ICDE World Conference, Tianjin, China.

    Google Scholar 

  • Kay, J., Reimann, P., Diebold, E., & Kummerfeld, B. (2013). MOOCs: So many learners, so much potential …. IEEE Intelligent Systems, 28(3), 70–77 (Kizilcic et al., 2013).

    Article  Google Scholar 

  • Kizilcec, R., Piech, C., & Schneider, E. (2013). Deconstructing disengagement: Analyzing learner subpopulations in massive open online courses. LAK’13 Leuven, Belgium.

    Google Scholar 

  • Kling, R., & Courtright, C. (2003). Group behavior and learning in electronic forums: A sociotechnical approach. The Information Society, 19, 221–235.

    Article  Google Scholar 

  • Kolowich, S. (2013, March 18). The professors behind the MOOC hype. The Chronicle of Higher Education. Retrieved from http://chronicle.com/article/The-Professors-Behind-the-MOOC/137905/.

  • Lackner, E., Ebner, M., & Khalil, M. (2015). MOOCs as granular systems: Design patterns to foster participant activity. eLearning Papers, 42, 28–37.

    Google Scholar 

  • Lin, Y.-L., Lin, H.-W., & Hung, T.-T. (2015). Value hierarchy for massive open online courses. Computers in Human Behaviour, 53, 408–418.

    Article  Google Scholar 

  • Littlejohn, A., Hood, N., Milligan, C., & Mustain, P. (2016). Learning in MOOCs: Motivations and self-regulated learning in MOOCs. The Internet and Higher Education, 29, 40–48. https://doi.org/10.1016/j.iheduc.2015.12.003.

    Article  Google Scholar 

  • Mak, S., Williams, R., & Mackness, J. (2010). Blogs and forums as communication and learning tools in a MOOC. In L. Dirckinck-Holmfeld, V. Hodgson, C. Jones, M. de Laat, D. McConnell, & T. Ryberg (Eds.), In Proceedings of the 7th International Conference on Networked Learning 2010 (pp. 275–284). Lancaster, UK: Lancaster University. Retrieved from https:// www.lancaster.ac.uk/fss/organisations/netlc/past/nlc2010/abstracts/PDFs/Mak.pdf.

  • Mamgain, N., Sharma, A., & Goyal, P. (2014). Learner’s perspective on video-viewing features offered by MOOC providers: Coursera and edX. Paper presented at the 2014 IEEE International Conference on MOOC, Innovation and Technology in Education (MITE). https://doi.org/10.1109/mite.2014.7020298.

  • Margaryan, A., Bianco, M., & Littlejohn, A. (2015). Instructional quality of massive open online courses (MOOCs). Computers & Education, 80, 77–83.

    Article  Google Scholar 

  • Morozov, E. (2014). The planning machine. The New Yorker, 13 October (www.newyorker.com/magazine/2014/10/13/planning-machine.

  • Morrison, D. (2014, January 18). Need-to-know-news: MOOC mentors for hire, Coursera’s MOC$s, edX shares MOOC data and more. Online Learning Insights. Retrieved from https://onlinelearninginsights.wordpress.com/2014/01/28/need-to-know-news-mooc-mentors-forhire-courseras-mocs-edx-shares-mooc-data-and-more/.

  • Perna, L., Ruby, A., Boruch, R., Wang, N., Scull, J., Ahmad, S., et al. (2014). Moving through MOOCs: Understanding the progression of users in massive open online courses. Education Researcher, 43(9), 421–432.

    Article  Google Scholar 

  • Rodriguez, C. (2012). MOOCs and the AI-Stanford like courses: Two successful and distinct course formats for massive open online courses. European Journal of Open, Distance and E-Learning. Retrieved from http://files.eric.ed.gov/fulltext/EJ982976.pdf.

  • Ross, J., Sinclair, C., Knox, J., & Macleod, H. (2014). Teacher experiences and academic identity: The missing components of MOOC pedagogy. Journal of Online Learning and Teaching, 10(1), 57.

    Google Scholar 

  • Scagnoli, N. (2012). Thoughts on instructional design for MOOCs. Retrieved from https://www.ideals.illinois.edu/bitstream/handle/2142/44835/Instructional%20Design%20of%20a%20MOOC.pdf.

  • Selwyn, N. (2014). Distrusting educational technology. London and New York: Routledge.

    Google Scholar 

  • Selwyn, N. (2016). Is technology good for education. Cambridge, UK: Polity Books.

    Google Scholar 

  • Shah, D. (2016). By the numbers: MOOCS in 2016. [Online]. Retrieved from: https://www.class-central.com/report/mooc-stats-2016/.

  • Sharples, M., & Domingue, J. (2016, September). The blockchain and kudos: A distributed system for educational record, reputation and reward. In European Conference on Technology Enhanced Learning (pp. 490–496). Springer International Publishing.

    Chapter  Google Scholar 

  • Sinha, T., Li, N., Jermann, P., & Dillenbourg, P. (2014). Capturing “attrition intensifying” structural traits from didactic interaction sequences of MOOC learners. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Workshop on Modeling Large Scale Social Interaction in Massively Open Online Courses (pp. 42–49). Retrieved from https://www.aclweb.org/anthology/W/W14/W14-41.pdf.

  • Tabba, Y., & Medouri, A. (2013). LASyM: A learning analytics system for MOOCs. International Journal of Advanced Computer Science and Applications, 4(5), 113–119.

    Google Scholar 

  • Tyler, K. (1939). Recent developments in radio education. The English Journal, 28(3), 193–199.

    Article  Google Scholar 

  • Warburton, S., & Mor, Y. (2015). Double loop design: Configuring narratives, patterns and scenarios in the design of technology enhanced learning. In M. Maina et al. (Eds.), The art and science of learning design (pp. 93–104). Rotterdam, The Netherlands: Sense Publishers.

    Chapter  Google Scholar 

  • Wen, M., Yang, D., & Rosé, C. P. (2014a). Linguistic reflections of student engagement in massive open online courses. In Proceedings of the International Conference on Weblogs and Social Media. Retrieved from http://www.cs.cmu.edu/~mwen/papers/icwsm2014-camera-ready.pdf.

  • Wen, M., Yang, D., & Rosé, C. P. (2014b). Sentiment analysis in MOOC discussion forums: What does it tell us? In Proceedings of Educational Data Mining. Retrieved from http://www.cs.cmu.edu/~mwen/papers/edm2014-camera-ready.pdf.

  • Yang, D., Wen, M., Kumar, A., Xing, E., & Rosé, C. (2014). Towards an integration of text and graph clustering methods as a lens for studying social interaction in MOOCs. International Review of Research in Open and Distributed Learning, 15(5). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1853/3083.

Download references

Acknowledgements

The authors wish to thank Vasudha Chaudhari of The Open University for comments and for proofing this chapter.

Author information

Authors and Affiliations

  1. Open University, Milton Keynes, UK

    Allison Littlejohn

  2. University of Auckland, Auckland, New Zealand

    Nina Hood

Authors
  1. Allison Littlejohn
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nina Hood
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Allison Littlejohn .

Rights and permissions

Reprints and permissions

Copyright information

© 2018 The Author(s)

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Littlejohn, A., Hood, N. (2018). Designing for Quality?. In: Reconceptualising Learning in the Digital Age. SpringerBriefs in Education(). Springer, Singapore. https://doi.org/10.1007/978-981-10-8893-3_5

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-8893-3_5

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-8892-6

  • Online ISBN: 978-981-10-8893-3

  • eBook Packages: EducationEducation (R0)

Publish with us

Policies and ethics

Search

Navigation