Playability Assessment for Educational Computer Games: Pilot Study for Model Development

  • Hasiah Mohamed
  • Azizah Jaafar
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8237)


This study used the partial least squares (PLS) and structural equation modeling (SEM) tool to examine evaluation constructs that are appropriate to evaluate educational computer games (ECG) by real users. The focus of the evaluation constructs are for evaluating ECG that still in the development process. There are five constructs that being derived from heuristic evaluation for ECG known as Playability Heuristic Evaluation for Educational Computer Games (PHEG) that being developed to evaluate ECG by expert evaluators. The constructs are interface (10), educational element (6), content (6), playability (7) and multimedia (8). Total numbers of items for the five constructs are 37. Statistical results confirm that all of the five constructs and 29 items are important to be included in evaluating ECG. The results, besides indicating the suitability of the PLS in statistical analysis, has also contributed to a better understanding of constructs needed in evaluating ECG that still in the development process that has not been tested. Findings are useful for game developers and educational technologist to have a validated model and questionnaire that can be used to evaluate ECG. The result of the evaluation using the proposed model can be used to enhance the ECG before it can be released. Limitations and suggestions for future research are also included.


playability assessment for educational computer games model interface usability 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Yee Leng, E., et al.: Computer games development and appreciative learning approach in enhancing students’ creative perception. Computers & Education 54(1), 146–161 (2010)CrossRefGoogle Scholar
  2. 2.
    Kato, P.M.: Video games in health care: Closing the gap. Review of General Psychology 14(2), 113 (2010)CrossRefGoogle Scholar
  3. 3.
    Robertson, J., Howells, C.: Computer game design: Opportunities for successful learning. Computers & Education 50(2), 559–578 (2008)CrossRefGoogle Scholar
  4. 4.
    Papastergiou, M.: Exploring the potential of computer and video games for health and physical education: A literature review. Computers & Education 53(3), 603–622 (2009)CrossRefGoogle Scholar
  5. 5.
    Mohamed, H., Jaafar, A.: Challenges in the evaluation of educational computer games. In: 2010 International Symposium on Information Technology (ITSim). IEEE (2010)Google Scholar
  6. 6.
    Hasiah, M.O., Azizah, J.: Heuristics evaluation in computer games. In: 2010 International Conference on Information Retrieval & Knowledge Management (CAMP). IEEE (2010)Google Scholar
  7. 7.
    Prensky, M.: Students as designers and creators of educational computer games: Who else? British Journal of Educational Technology 39(6), 1004–1019 (2008)CrossRefGoogle Scholar
  8. 8.
    Oblinger, D.G.: The next generation of educational engagement. Journal of Interactive Media in Education (2004)Google Scholar
  9. 9.
    Hogle, J.G.: Considering games as cognitive tools: in search of effective “edutainment”. ERIC Clearinghouse (1996)Google Scholar
  10. 10.
    Barab, S., et al.: Making learning fun: Quest Atlantis, a game without guns. Educational Technology Research and Development 53(1), 86–107 (2005)CrossRefGoogle Scholar
  11. 11.
    Kiili, K.: Content creation challenges and flow experience in educational games: The IT-Emperor case. The Internet and Higher Education 8(3), 183–198 (2005)CrossRefGoogle Scholar
  12. 12.
    Van Eck, R.: Digital game-based learning: It’s not just the digital natives who are restless. EDUCAUSE Review 41(2), 16 (2006)Google Scholar
  13. 13.
    Prensky, M.: Digital game-based learning. McGraw-Hill, New York (2001)Google Scholar
  14. 14.
    Pohl, M., et al.: Designing game based learning-a participatory approach. In: World Conference on Educational Multimedia, Hypermedia and Telecommunications (2008)Google Scholar
  15. 15.
    Dondlinger, M.J.: Educational video game design: A review of the literature. Journal of Applied Educational Technology 4(1), 21–31 (2007)Google Scholar
  16. 16.
    Gee, J.P.: What video games have to teach us about learning and literacy. Palgrave Macmillan, New York (2003)Google Scholar
  17. 17.
    Gee, J.P.: Are video games good for learning? (2006), (cited Oktober 18, 2008)
  18. 18.
    Cordova, D.I., Lepper, M.R.: Intrinsic motivation and the process of learning: Beneficial effects of contextualization, personalization, and choice. Journal of Educational Psychology 88, 715–730 (1996)CrossRefGoogle Scholar
  19. 19.
    Lopez-Morteo, G., López, G.: Computer support for learning mathematics: A learning environment based on recreational learning objects. Computers & Education 48(4), 618–641 (2007)CrossRefGoogle Scholar
  20. 20.
    Tüzün, H., et al.: The effects of computer games on primary school students’ achievement and motivation in geography learning. Computers & Education 52(1), 68–77 (2009)CrossRefGoogle Scholar
  21. 21.
    von der Heiden, B., et al.: Learning by Playing: Potential of Serious Games to Increase Intellectual Capital. In: Enabling Innovation, pp. 375–388. Springer (2011)Google Scholar
  22. 22.
    Hillen, S.A., Breuer, K.U., Tennyson, R.D.: Gaming and learning: Theory, research, and practice. In: Challenges Facing Contemporary Didactics: Diversity of Students and the Role of New Media in Teaching and Learning, p. 185 (2011)Google Scholar
  23. 23.
    Thomas, P.J., Meech, J.F., Macredie, R.D.: A framework for the development of information appliances. ACM SIGICE Bulletin 21(1), 15–19 (1995)CrossRefGoogle Scholar
  24. 24.
    Amory, A., et al.: Computer games as a learning resource. In: World Conference on Educational Multimedia, Hypermedia and Telecommunications (1998)Google Scholar
  25. 25.
    Garris, R., Ahlers, R., Driskell, J.E.: Games, motivation, and learning: A research and practice model. Simulation & Gaming 33(4), 441–467 (2002)CrossRefGoogle Scholar
  26. 26.
    Malone, T.W., Lepper, M.R.: Making learning fun: A taxonomy of intrinsic motivations for learning. Aptitude, Learning, and Instruction 3, 223–253 (1987)Google Scholar
  27. 27.
    Nielsen, J., Molich, R.: Heuristic evaluation of user interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Empowering People. ACM (1990)Google Scholar
  28. 28.
    Nielsen, J.: Heuristic evaluation. In: Nielsen, J., Mark, R.L. (eds.) Usability Inspection Methods. John Wiley & Sons, New York (1994)Google Scholar
  29. 29.
    Hvannberg, E.T., Law, E.L.-C., Lárusdóttir, M.K.: Heuristic evaluation: Comparing ways of finding and reporting usability problems. Interacting with Computers 19(2), 225–240 (2007)CrossRefGoogle Scholar
  30. 30.
    Desurvire, H., Wiberg, C.: Master of the game: assessing approachability in future game design. In: Proceedings of ACM CHI 2008 Conference on Human Factors in Computing Systems (2008)Google Scholar
  31. 31.
    Ling, C., Salvendy, G.: Optimizing Heuristic Evaluation Process in E-Commerce: Use of the Taguchi Method. International Journal of Human-Computer Interaction 22(3), 271–287 (2007)CrossRefGoogle Scholar
  32. 32.
    Ling, C., Salvendy, G.: Effect of evaluators’ cognitive style on heuristic evaluation: Field dependent and field independent evaluators. International Journal of Human-Computer Studies 67(4), 382–393 (2009)CrossRefGoogle Scholar
  33. 33.
    Nielsen, J., Molich, R.: Heuristics Evaluation of User Interfaces. In: Proceedings of Human Computer Interaction (1994)Google Scholar
  34. 34.
    Squires, D., Preece, J.: Usability and learning: Evaluating the potential of educational software. Computers & Education 27(1), 15–22 (1996)CrossRefGoogle Scholar
  35. 35.
    Squires, D.: Usability and Educational Software Design: Special Issue of Interacting with Computers. Interacting with Computers 11(5), 463–466 (1999)CrossRefGoogle Scholar
  36. 36.
    Squires, D., Preece, J.: Predicting quality in educational software: Evaluating for learning, usability and the synergy between them. Interacting with Computers 11(5), 467–483 (1999)CrossRefGoogle Scholar
  37. 37.
    Reeves, T., et al.: Usability and instructional design heuristics for e-learning evaluation. In: World Conference on Educational Multimedia, Hypermedia and Telecommunications. AACE, Charlottesville (2002)Google Scholar
  38. 38.
    Evans, C., Sabry, K.: Evaluation of the interactivity of web-based learning systems: Principles and process. Innovations in Education and Teaching International 40(1), 89–99 (2003)CrossRefGoogle Scholar
  39. 39.
    Albion, P.: Heuristic evaluation of educational multimedia: from theory to practice. In: Proceedings ASCILITE 1999: 16th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education: Responding to Diversity. Australasian Society for Computers in Learning in Tertiary Education, ASCILITE (1999)Google Scholar
  40. 40.
    Ssemugabi, S., de Villiers, R.: A comparative study of two usability evaluation methods using a web-based e-learning application (2007)Google Scholar
  41. 41.
    Shee, D.Y., Wang, Y.-S.: Multi-criteria evaluation of the web-based e-learning system: A methodology based on learner satisfaction and its applications. Computers & Education 50(3), 894–905 (2008)CrossRefGoogle Scholar
  42. 42.
    Hsu, S.H., Wen, M.-H., Wu, M.-C.: Exploring user experiences as predictors of MMORPG addiction. Computers & Education 53(3), 990–999 (2009)CrossRefGoogle Scholar
  43. 43.
    Malone, T.W.: Heuristics for designing enjoyable user interfaces: Lessons from computer games. In: Thomas, J.C., Schneider, M.L. (eds.) Human Factors in Computing Systems. Ablex Publishing Corporation, Norwood (1982)Google Scholar
  44. 44.
    Federoff, M.A.: Heuristics and Usability Guidelines for the Creation and Evaluation of Fun in Video Games. Indiana University (2002)Google Scholar
  45. 45.
    Desurvire, H., Caplan, M., Toth, J.A.: Using heuristics to evaluate the playability of games. In: Computer Human Interaction (CHI 2004), Vienna, Austria (2004)Google Scholar
  46. 46.
    Rieber, L.P., et al.: The role of meaning in interpreting graphical and textual feedback during a computer-based simulation. Computers & Education 27(1), 45–58 (1996)CrossRefGoogle Scholar
  47. 47.
    Mayer, R.E., Moreno, R.: A cognitive theory of multimedia learning: Implications for design principles. In: Annual Meeting of the ACM SIGCHI Conference on Human Factors in Computing Systems, Los Angeles, CA (1998)Google Scholar
  48. 48.
    Barr, P., Noble, J., Biddle, R.: Video game values: Human-computer interaction and games. Interacting with Computers 19(2), 180–195 (2007)CrossRefGoogle Scholar
  49. 49.
    Sekaran, U., Bougie, R.: Research methods for business: a skill building approach. Wiley, UK (2010)Google Scholar
  50. 50.
    Hair, J., Babin, B.J., Anderson, R.E.: Multivariate Data Analysis: A global perspective. Pearson, Upper Saddle River (2010)Google Scholar
  51. 51.
    Barclay, D., Higgins, C., Thompson, R.: The partial least squares (PLS) approach to causal modeling: personal computer adoption and use as an illustration. Technology Studies 2(2), 285–309 (1995)Google Scholar
  52. 52.
    Fornell, C., Larcker, D.F.: Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 39–50 (1981)Google Scholar
  53. 53.
    Nunnally, J.: Psychometric theory. McGraw-Hill, New York (1978)Google Scholar
  54. 54.
    Churchill Jr., G.A.: A paradigm for developing better measures of marketing constructs. Journal of Marketing Research, 64–73 (1979)Google Scholar
  55. 55.
    Compeau, D.R., Higgins, C.A.: Computer self-efficacy: Development of a measure and initial test. Mis Quarterly, 189–211 (1995)Google Scholar
  56. 56.
    Hair, J.F., et al.: Multivariate data analysis. Prentice Hall, Upper Saddle River (1998)Google Scholar
  57. 57.
    Nunnally, J.C., Bernstein, I.H.: Psychometric theory. McGraw, New York (1991, 1994)Google Scholar
  58. 58.
    Gefen, D., Straub, D.W., Boudreau, M.-C.: Structural equation modeling and regression: Guidelines for research practice. In: Communications of the Association for Information Systems. Citeseer (2000)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2013

Authors and Affiliations

  • Hasiah Mohamed
    • 1
  • Azizah Jaafar
    • 2
  1. 1.Faculty of Computer & Mathematical SciencesUniversiti Teknologi MARADungunMalaysia
  2. 2.Institute of Visual InformaticsUniversiti Kebangsaan MalaysiaBangiMalaysia

Personalised recommendations