Skip to main content
SpringerLink
Log in

Guidelines to Evaluate the Usability and User Experience of Learning Support Platforms: A Systematic Review

  • Conference paper
  • First Online:
Human-Computer Interaction (HCI-COLLAB 2019)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1114))

Included in the following conference series:

Abstract

Nowadays, the usability and the user experience, are important aspects for the success of any software product. In the educational domain, Learning Support Platforms are not the exception, and these quality attributes become essential to guarantee an appropriate teaching process. For this reason, in this study, we present a systematic literature review whose purpose was to identify the characteristics that these types of systems must meet to obtain usable products. In the same way, the features that contribute to the assurance of a satisfying user experience have been cataloged. The purpose of this study is to establish the basis towards the elaboration of a future framework to quantify the level of usability and user experience of learning support platforms. The systematic search retrieved a total of 105 studies, from which 23 were selected as relevant. A set of 15 sub-attributes with their corresponding guidelines were identified to serve as a guide to design graphical user interfaces in a correct way in this type of software systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Penha, M., Correia, W., Soares, M., Campos, F., Barros, M.: Ergonomic evaluation of usability with users – application of the technique of cooperative evaluation. In: Marcus, A. (ed.) DUXU 2013. LNCS, vol. 8012, pp. 379–388. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39229-0_41

    Chapter  Google Scholar 

  2. Ilyas, M., Kadir, K.A., Adnan, Z.: Demystifying the learning management system (LMS): journey from e-learning to the strategic role. Eur. J. Bus. Manag. 9, 12–18 (2017)

    Google Scholar 

  3. Cavus, N., Alhih, M.S.: Learning management systems use in science education. Procedia - Soc. Behav. Sci. 143, 517–520 (2014). https://doi.org/10.1016/j.sbspro.2014.07.429

    Article  Google Scholar 

  4. Westfall, B.: Learning Management System User Report - 2016. SoftwareAdvice.com (2016). https://www.softwareadvice.com/resources/lms-user-trends-2016/

  5. Kumar, R., Hasteer, N.: Evaluating usability of a web application: a comparative analysis of open-source tools. In: Proceedings of the 2nd International Conference on Communication and Electronics Systems (ICCES 2017), Coimbatore, India, pp. 350–354 (2017). https://doi.org/10.1109/CESYS.2017.8321296

  6. Hasan, L.A., Al-Sarayreh, K.T.: An integrated measurement model for evaluating usability attributes. In: Proceedings of the International Conference on Intelligent Information Processing, Security and Advanced Communication, pp. 1–6. ACM, Batna (2015). https://doi.org/10.1145/2816839.2816861

  7. Granollers, T.: Usability evaluation with heuristics. new proposal from integrating two trusted sources. In: Marcus, A., Wang, W. (eds.) DUXU 2018. LNCS, vol. 10918, pp. 396–405. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91797-9_28

    Chapter  Google Scholar 

  8. Paz, F., Paz, F.A., Sánchez, M., Moquillaza, A., Collantes, L.: Quantifying the usability through a variant of the traditional heuristic evaluation process. In: Marcus, A., Wang, W. (eds.) DUXU 2018. LNCS, vol. 10918, pp. 496–508. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91797-9_36

    Chapter  Google Scholar 

  9. Granollers, T.: Validación experimental de un conjunto heurístico para evaluaciones de UX de sitios web de comercio-e. In: Proceedings of the 2016 IEEE 11th Colombian Computing Conference (CCC 2016), Popayan, Colombia, pp. 1–8 (2016). https://doi.org/10.1109/ColumbianCC.2016.7750783

  10. Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., Linkman, S.: Systematic literature reviews in software engineering – A systematic literature review. Inf. Softw. Technol. 51, 7–15 (2009). https://doi.org/10.1016/j.infsof.2008.09.009

    Article  Google Scholar 

  11. Petticrew, M., Roberts, H.: Systematic Reviews in the Social Sciences: A Practical Guide. Blackwell Publishing, Hoboken (2005). ISBN 978-1-405-12110-1

    Google Scholar 

  12. Hedegaard, S., Simonsen, J.G.: Extracting usability and user experience information from online user reviews. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2089–2098. ACM, Paris (2013). https://doi.org/10.1145/2470654.2481286

  13. Al-Faries, A., Al-Khalifa, H.S., Al-Razgan, M.S., Al-Duwais, M.: Evaluating the accessibility and usability of top Saudi e-government services. In: Proceedings of the 7th International Conference on Theory and Practice of Electronic Governance, pp. 60–63. ACM, Seoul (2013). https://doi.org/10.1145/2591888.2591898

  14. Dias, A.L., Fortes, R.P.d.M., Masiero, P.C.: HEUA: a heuristic evaluation with usability and accessibility requirements to assess web systems. In: Proceedings of the 11th Web for All Conference, pp. 1–4. ACM, Seoul (2014). https://doi.org/10.1145/2596695.2596706

  15. Hovde, M.R.: Effective user experience in online technical communication courses: employing multiple methods within organizational contexts to assess usability. In: Proceedings of the 33rd Annual International Conference on the Design of Communication, pp. 1–5. ACM, Limerick (2015). https://doi.org/10.1145/2775441.2775453

  16. Choma, J., Zaina, L.A.M., Silva, T.S.d.: Working beyond technical aspects: an approach for driving the usability inspection adding the perspective of user experience. In: Proceedings of the 34th ACM International Conference on the Design of Communication, pp. 1–10. ACM, Silver Spring (2016). https://doi.org/10.1145/2987592.2987607

  17. Quiñones, D., Rusu, C., Rusu, V.: A methodology to develop usability/user experience heuristics. Comput. Stand. Interfaces 59, 109–129 (2018). https://doi.org/10.1016/j.csi.2018.03.002

    Article  Google Scholar 

  18. Sagar, K., Saha, A.: Qualitative usability feature selection with ranking: a novel approach for ranking the identified usability problematic attributes for academic websites using data-mining techniques. Hum.-Centric Comput. Inf. Sci. 7, 1–24 (2017). https://doi.org/10.1186/s13673-017-0111-8

    Article  Google Scholar 

  19. Hasan, L.: Usability problems on desktop and mobile interfaces of the Moodle learning management system (LMS). In: Proceedings of the 2018 International Conference on E-Business and Applications, pp. 69–73. ACM, Da Nang (2018). https://doi.org/10.1145/3194188.3194192

  20. AlRoobaea, R.S., Al-Badi, A.H., Mayhew, P.J.: A framework for generating a domain specific inspection evaluation method: a comparative study on social networking websites. In: Proceedings of the 2013 Science and Information Conference (SAI 2013), London, UK, pp. 757–767 (2013). ISBN 978-0-9893193-0-0

    Google Scholar 

  21. Gordillo, A., Barra, E., Aguirre, S., Quemada, J.: The usefulness of usability and user experience evaluation methods on an e-Learning platform development from a developer’s perspective: a case study. In: Proceedings of the 2014 IEEE Frontiers in Education Conference (FIE 2014), Madrid, Spain, pp. 1–8 (2014). https://doi.org/10.1109/FIE.2014.7044340

  22. Iman, M.R.H., Rasoolzadegan, A.: Quantitative evaluation of software usability with a fuzzy expert system. In: Proceedings of the 2015 5th International Conference on Computer and Knowledge Engineering (ICCKE 2015), Mashhad, Iran, pp. 325–330 (2015). https://doi.org/10.1109/ICCKE.2015.7365850

  23. Mtebe, J.S., Kissaka, M.M.: Heuristics for evaluating usability of Learning Management Systems in Africa. In: Proceedings of the 2015 IST-Africa Conference, Lilongwe, Malawi, pp. 1–13 (2015). https://doi.org/10.1109/ISTAFRICA.2015.7190521

  24. Kabir, M.A., Rehman, M.U., Majumdar, S.I.: An analytical and comparative study of software usability quality factors. In: Proceedings of the 2016 7th IEEE International Conference on Software Engineering and Service Science (ICSESS 2016), Beijing, China, pp. 800–803 (2016). https://doi.org/10.1109/ICSESS.2016.7883188

  25. Ivanović, M., Putnik, Z., Komlenov, Ž.: Usability and privacy aspects of Moodle: students’ and teachers’ perspective. Informatica 37, 221–230 (2013)

    Google Scholar 

  26. Sabri, N.M., Mohamed, H., Soon, G.Y., Yusof, Y.m.H.: A quantitative approach in the usability evaluation of a courseware. J. Next Gener. Inf. Technol. 4, 29–38 (2013). https://doi.org/10.4156/jnit.vol4.issue2.4

  27. Deraniyagala, R.L., Amdur, R.J., Boyer, A.L., Kaylor, S.: Usability study of the EduMod eLearning Program for contouring nodal stations of the head and neck. Pract. Radiat. Oncol. 5, 169–175 (2015). https://doi.org/10.1016/j.prro.2014.10.008

    Article  Google Scholar 

  28. Junus, I.S., Santoso, H.B., Isal, Y.K., Utomo, A.Y.: Usability evaluation of the student centered e-Learning environment. Int. Rev. Res. Open Distance Learn. 16, 62–82 (2015). https://doi.org/10.19173/irrodl.v16i4.2175

    Article  Google Scholar 

  29. Murillo, B., Vargas, S., Moquillaza, A., Fernández, L., Paz, F.: Usability testing as a complement of heuristic evaluation: a case study. In: Marcus, A., Wang, W. (eds.) DUXU 2017. LNCS, vol. 10288, pp. 434–444. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58634-2_32

    Chapter  Google Scholar 

  30. Emang, D.W.A.B., Lukman, R.N.I.R., Kamarulzaman, M.I.S., Zaaba, Z.F.: Usability studies on e-learning platforms: preliminary study in USM. In: Proceedings of the 2nd International Conference on Applied Science and Technology (ICAST 2017), Kedah, Malaysia, vol. 1891, pp. 1–8 (2017). https://doi.org/10.1063/1.5005373

  31. Aparna, S.S., Baseer, K.K.: SIRIUS-WUEP: a heuristic-based framework for measuring and evaluating web usability in model-driven web development. In: Satapathy, S., Govardhan, A., Raju, K., Mandal, J. (eds.) Emerging ICT for Bridging the Future - Proceedings of the 49th Annual Convention of the Computer Society of India (CSI) Volume 1. Advances in Intelligent Systems and Computing, vol. 337, pp. 303–310. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-13728-5_34

    Chapter  Google Scholar 

  32. Eason, K.D.: Towards the experimental study of usability. Behav. Inf. Technol. 3, 133–143 (1984). https://doi.org/10.1080/01449298408901744

    Article  Google Scholar 

  33. Nielsen, J.: Usability Engineering. Morgan Kaufmann, Burlington (1993). ISBN 0125184050

    Chapter  Google Scholar 

  34. ISO: Ergonomic requirements for office work with visual display terminals (VDTs) – Part 11: Guidance on usability - ISO 9241-11:1998. International Organization for Standardization (1998)

    Google Scholar 

  35. ISO: Software engineering – Product quality – Part 1: Quality model - ISO/IEC 9126:2001. International Organization for Standardization (2001)

    Google Scholar 

  36. Reeves, T.C., et al.: Usability and instructional design heuristics for E-Learning evaluation. In: Proceedings of the 2002 World Conference on Educational Multimedia, Hypermedia & Telecommunications (ED-MEDIA 2002), Denver, Colorado, USA, pp. 2–8 (2002). https://eric.ed.gov/?id=ED477084

  37. Folmer, E., van Gurp, J., Bosch, J.: A framework for capturing the relationship between usability and software architecture. Softw. Process: Improv. Pract. 8, 67–87 (2003). https://doi.org/10.1002/spip.171

    Article  Google Scholar 

  38. Abran, A., Khelifi, A., Suryn, W., Seffah, A.: Usability meanings and interpretations in ISO standards. Softw. Qual. J. 11, 325–338 (2003). https://doi.org/10.1023/A:1025869312943

    Article  Google Scholar 

  39. Rosato, J., Dodds, C., Laughlin, S.: Usability of course management systems by students. Department of Computer Information Systems/Computer Science, College of Scholastica, Duluth (2006)

    Google Scholar 

  40. Mehlenbacher, B., et al.: Usable e-learning: a conceptual model for evaluation and design. In: 11th International Conference on Human-Computer Interaction (HCI International 2005), Las Vegas, NV, USA, pp. 1–10 (2005). http://citeseerx.ist.psu.edu/viewdoc/citations?doi=10.1.1.84.331

  41. Dringus, L.P., Cohen, M.S.: An adaptable usability heuristic checklist for online courses. In: Proceedings Frontiers in Education 35th Annual Conference, Indianopolis, IN, USA (2005). https://doi.org/10.1109/FIE.2005.1611918

  42. Ardito, C., et al.: An approach to usability evaluation of e-learning applications. Univ. Access Inf. Soc. 4, 270–283 (2006). https://doi.org/10.1007/s10209-005-0008-6

    Article  Google Scholar 

  43. Seffah, A., Donyaee, M., Kline, R.B., Padda, H.K.: Usability measurement and metrics: a consolidated model. Softw. Qual. J. 14, 159–178 (2006). https://doi.org/10.1007/s11219-006-7600-8

    Article  Google Scholar 

  44. Nielsen, J., Loranger, H.: Prioritizing Web Usability. New Riders Publishing, Thousand Oaks (2006). ISBN 0321350316

    Google Scholar 

  45. Bevan, N.: Classifying and selecting UX and usability measures. In: Proceedings of the Valid Useful User Experience Measurement (VUUM 2008), pp. 1–6. University of Iceland, Reykjavik (2008)

    Google Scholar 

  46. Ketola, P., Roto, V.: Exploring user experience measurement needs. In: Proceedings of the Valid Useful User Experience Measurement (VUUM 2008), pp. 23–26. University of Iceland, Reykjavik (2008). https://doi.org/10.4018/jthi.2009070104

    Article  Google Scholar 

  47. Zaharias, P., Poylymenakou, A.: Developing a usability evaluation method for e-learning applications: beyond functional usability. Int. J. Hum.-Comput. Interact. 25, 75–98 (2009). https://doi.org/10.1080/10447310802546716

    Article  Google Scholar 

  48. Giannakos, M.N.: A combinational evaluation method of computer applications. J. Basic Appl. Sci. 1, 240–242 (2009)

    Google Scholar 

  49. Al-Khalifa, H.S.: Heuristic evaluation of the usability of e-government websites: a case from Saudi Arabia. In: Proceedings of the 4th International Conference on Theory and Practice of Electronic Governance, pp. 238–242. ACM, Beijing (2010). http://doi.acm.org/10.1145/1930321.1930370

  50. Al-sarrayrih, H.S., Knipping, L., Zorn, E.: Evaluation of a MOODLE based Learning Management System applied at Berlin Institute of Technology based on ISO-9126. In: 13th International Conference on Interactive Computer aided Learning (ICL2010), Hasselt, Belgium, pp. 1–8 (2010)

    Google Scholar 

  51. Giannakos, M.N.: The evaluation of an E-Learning Web-based platform. In: Proceedings of the 2nd International Conference on Computer Supported Education (CSEDU 2010), Valencia, Spain, pp. 433–438 (2010). https://doi.org/10.5220/0002799504330438

  52. ISO: Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – System and software quality models - ISO/IEC 25010:2011. International Organization for Standardization (2011)

    Google Scholar 

  53. Zaharias, P.: Heuristic evaluation of e-learning courses: a comparative analysis of two e-learning heuristic sets. Campus-Wide Inf. Syst. 29, 45–60 (2012). ISSN 1065-0741

    Article  Google Scholar 

  54. Ghirardini, B.: E-Learning Methodologies: A Guide for Designing and Developing E-Learning Courses. Food and Agriculture Organization of the United Nations - FAO, USA (2012)

    Google Scholar 

  55. Kumar Dubey, S., Rana, A., Sharma, A.: Usability evaluation of object oriented software system using fuzzy logic approach. Int. J. Comput. Appl. 43, 1–6 (2012). https://doi.org/10.5120/6208-8778

    Article  Google Scholar 

  56. Torrente, M.C.S., Prieto, A.B.M., Gutiérrez, D.A., de Sagastegui, M.E.A.: Sirius: a heuristic-based framework for measuring web usability adapted to the type of website. J. Syst. Softw. 86, 649–663 (2013). https://doi.org/10.1016/j.jss.2012.10.049

    Article  Google Scholar 

  57. Şenol, L., Gecili, H., Durdu, P.O.: Usability evaluation of a Moodle based learning management system. In: Proceedings of the World Conference on Educational Media and Technology (EdMedia 2014), Tampere, Finland, pp. 850–858 (2014). ISBN 978-1-939797-08-7

    Google Scholar 

  58. Thuseethan, S., Achchuthan, S., Kuhanesan, S.: Usability evaluation of learning management systems in Sri Lankan Universities. Adv. Hum. Comput. Interact. 15, 1–13 (2015)

    Google Scholar 

  59. Gupta, D., Ahlawat, A., Sagar, K.: A critical analysis of a hierarchy based Usability Model. In: Proceedings of the 2014 International Conference on Contemporary Computing and Informatics (IC3I 2014), Mysore, India, pp. 255–260 (2014). https://doi.org/10.1109/IC3I.2014.7019810

  60. Paz, F., Paz, F.A., Villanueva, D., Pow-Sang, J.A.: Heuristic evaluation as a complement to usability testing: a case study in web domain. In: Proceedings of the 12th International Conference on Information Technology - New Generations (ITNG 2015), Las Vegas, NV, USA, pp. 546–551 (2015). https://doi.org/10.1109/ITNG.2015.92

  61. Issa, T., Isaias, P.: Sustainable Design: HCI, Usability and Environmental Concerns. Springer, Heidelberg (2015). https://doi.org/10.1007/978-1-4471-6753-2

    Book  Google Scholar 

  62. Rogers, Y., Preece, J., Sharp, H.: Interaction Design: Beyond Human-Computer Interaction. Wiley, Hoboken (2015). ISBN 978-1-119-02075-2

    Google Scholar 

  63. Farmanesh, P., Samani, A.A.: Heuristic evaluation of the usability of Learning Management System (Moodle) at Eastern Mediterranean University, pp. 1–25 (2015). https://doi.org/10.5281/zenodo.59517

Download references

Acknowledgement

This study is highly supported by the Section of Informatics Engineering of the Pontifical Catholic University of Peru (PUCP) – Peru, and the “HCI, Design, User Experience, Accessibility & Innovation Technologies” Research Group (HCI-DUXAIT). HCI-DUXAIT is a research group of PUCP.

Author information

Authors and Affiliations

  1. Pontificia Universidad Católica del Perú, San Miguel, Lima 32, Peru

    Juan Salas, Alberto Chang, Lourdes Montalvo, Almendra Núñez, Max Vilcapoma, Arturo Moquillaza, Braulio Murillo & Freddy Paz

  2. Universidad San Ignacio de Loyola, La Molina, Lima 12, Peru

    Arturo Moquillaza

Authors
  1. Juan Salas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alberto Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lourdes Montalvo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Almendra Núñez
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Max Vilcapoma
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Arturo Moquillaza
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Braulio Murillo
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Freddy Paz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Juan Salas .

Editor information

Editors and Affiliations

  1. Corporación Universitaria Comfacauca - Unicomfacauca and Universidad del Cauca, Popayán, Colombia

    Pablo H. Ruiz

  2. Corporación Universitaria Comfacauca - Unicomfacauca and Universidad del Cauca, Popayán, Colombia

    Vanessa Agredo-Delgado

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Salas, J. et al. (2019). Guidelines to Evaluate the Usability and User Experience of Learning Support Platforms: A Systematic Review. In: Ruiz, P., Agredo-Delgado, V. (eds) Human-Computer Interaction. HCI-COLLAB 2019. Communications in Computer and Information Science, vol 1114. Springer, Cham. https://doi.org/10.1007/978-3-030-37386-3_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-37386-3_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-37385-6

  • Online ISBN: 978-3-030-37386-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation