Skip to main content
SpringerLink
Log in
Book cover

Entertainment Computing and Serious Games pp 284–305Cite as

The Experimental Method as an Evaluation Tool in Serious Games Research and Development

  • Chapter
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 9970))

Abstract

This chapter aims to provide the reader with basic knowledge about the experiment as a general method that can be applied towards serious games research and evaluation. It explains the main terms and rules of the experimental design, as well as points out the main risks and difficulties to avoid. The chapter also explains the differences in possible conclusions between true experiments, quasi-experiments, and correlational studies.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Adams, K.A., Lawrence, E.K.: Research Methods, Statistics and Applications. SAGE Publications, London (2014)

    Google Scholar 

  2. Adejumobi, B., Franck, N., Janzen, M.: Designing and testing a racing car serious game module. In: Ma, M., Oliveira, M.F., Baalsrud Hauge, J. (eds.) SGDA 2014. LNCS, vol. 8778, pp. 192–198. Springer, Heidelberg (2014)

    Google Scholar 

  3. Anderson, C.A., Dill, K.E.: Video games and aggressive thoughts, feelings, and behavior in the laboratory and in life. J. Person. Soc. Psychol. 78(4), 772–790 (2000)

    Article  Google Scholar 

  4. Anguera, J.A., Boccanfuso, J., Rintoul, J.L., Al-Hashimi, O., Faraji, F., Janowich, J., Kong, E., Larraburo, Y., Rolle, C., Johnston, E., Gazzaley, A.: Video game training enhances cognitive control in older adults. Nature 501, 97–101 (2013)

    Article  Google Scholar 

  5. Baek, Y., Ko, R., Marsh, T. (eds.): Trends and Applications of Serious Gaming and Social Media. Gaming Media and Social Effects. Springer, New York (2014)

    Google Scholar 

  6. Bidarra, R., Gambon, D., Kooij, R., Nagel, D., Schutjes, M., Tziouvara, I.: Gaming at the Dentist’s–serious game design for pain and discomfort distraction. In: Schouten, B., Fedtke, S., Bekker, T., Schijven, M., Gekker, A. (eds.) Games for Health 2013. Proceeding of the 3rd Conference on Gaming and Playful Interaction in Healthcare, pp. 207–215. Springer, Heidelberg (2013)

    Google Scholar 

  7. Boinodiris, P., Fingar, P.: Serious Games for Business: Using Gamification to Fully Engage Customers, Employees and Partners. Meghan-Kiffer Press, Tampa (2014)

    Google Scholar 

  8. Botella, C., Breton-Lópeza, J., Queroa, S., Bañosb, R.M., García-Palaciosa, A., Zaragozac, I., Alcanizc, M.: Treating cockroach phobia using a serious game on a mobile phone and augmented reality exposure: a single case study. Comput. Hum. Behav. 27(1), 217–227 (2011). Current Research Topics in Cognitive Load Theory. Third International Cognitive Load Theory Conference

    Article  Google Scholar 

  9. Brom, C., Buchtová, M., Šisler, V., Děchtěrenko, F., Palme, R., Glenk, L.M.: Flow, social interaction anxiety and salivary cortisol responses in serious games: a quasi-experimntal study. Comput. Educ. 79, 69–100 (2014)

    Article  Google Scholar 

  10. Bushman, B.J., Anderson, C.: Violent video games and hostile expectations: a test of the general aggression model. Pers. Soc. Psychol. Bull. 28(12), 1679–1686 (2002)

    Article  Google Scholar 

  11. Campbell, D.T., Cook, D.T.: Quasy-Experimental Design and Analysis Issues for Field Setting. Rand McNally, Chicago (1979)

    Google Scholar 

  12. Casti, J.L.: Reality Rules, Picturing the World in Mathematics–The Fundamentals, vol. I. Wiley, New York (1997)

    MATH  Google Scholar 

  13. Choo, A., Tong, X., Gromala, D., Hollander, A.: Virtual reality and mobius floe: cognitive distraction as non-pharmacological analgesic for pain management. In: Schouten, B., Fedtke, S., Schijven, M., Vosmeer, M., Gekker, A. (eds.) Games for Health 2014 Proceeding of the 4th conference on gaming and playful interaction in healthcare, pp. 8–12. Springer Fachmedien Wiesbaden, Heidelberg (2014)

    Google Scholar 

  14. Adams, K.A.: Cram101 Textbooks Reviews, Just the facts101 Textbook Key Facts: Research Methods, Statistics, and Applications. Content Technologies (2014). http://cram101.com

  15. De Freitas, S., Liarokapis, F.: Serious games: a new paradigm for education? In: Ma, M., Oikonomou, A., Jain, L.C. (eds.) Serious Games and Edutainment Applications, pp. 9–23. Springer, London (2011)

    Chapter  Google Scholar 

  16. De Gloria, A., Bellotti, F., Berta, R.: Serious games for education and training. Int. J. Serious Games. 1(1). http://dx.doi.org/10.17083/ijsg.v1i1.11

  17. Djaouti, D., Alvarez, J., Jessel, J., Rampnoux, O.: Origins of Serious Games. In: Ma, M., Oikonomou, A., Jain, L.C. (eds.) Serious Games and Edutainment Applications, pp. 25–43. Springer, London (2011)

    Chapter  Google Scholar 

  18. Essential facts about the computer and video game industry. 2016 Sales, Demographic and Usage Data. ESA (2016). http://www.essentialfacts.theesa.com/Essential-Facts-2016.pdf

  19. Ethical Principles of Psychologisys and Code of Conduct. APA (2010). http://www.apa.org/ethics/code/

  20. Experiment. In: Merriam-Webster.com. Merriam-Webster. http://www.merriam-webster.com/dictionary/experiment

  21. Ferguson, C., Kilburn, J.: Much ado about nothing: the misestimation and overinterpretation of violent video game effects in eastern and western nations: a comment on Anderson, et al. Psychol. Bull. 136(2), 174–178 (2010)

    Article  Google Scholar 

  22. Foster, J., Barkus, E., Yavorsky, C.: Understanding and Using Advanced Statistics: A Practical Guide for Students. SAGE Puclications, London, Thousand Oaks (2006)

    Book  Google Scholar 

  23. Girard, C., Ecalle, J., Magnan, A.: Serious games as new educational tools: how effective are they? a meta-analysis of recent studies. J. Comput. Assist. Learn. 29(3), 207–219 (2013)

    Article  Google Scholar 

  24. Goodwin, C.J.: Research in Psychology Methods and Design. Wiley, New York (2010)

    Google Scholar 

  25. Hess, M., Wiemeyer, J., Hamacher, K., Goesele, M.: Serious games for solving protein sequence alignments - combining citizen science and gaming. In: Göbel, S., Wiemeyer, J. (eds.) GameDays 2014. LNCS, vol. 8395, pp. 175–185. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  26. Hypothesis. In: Merriam-Webster.com. Merriam-Webster. http://www.merriam-webster.com/dictionary/hypothesis

  27. Kantowitz, B.H., Roediger, H.L., Elmes, D.G.: Experimental Psychlogy, vol. III. Wadsworth, Belmont (2009)

    Google Scholar 

  28. Kaufman, S.B.: Sex differences in mental rotation and spatial visualization ability: can they be accounted for by differences in working memory capacity? Intelligence 35, 211–223 (2007)

    Article  Google Scholar 

  29. King, D.L., Delfabbro, P., Griffiths, M.D.: The psychological study of video game players: methodological challenges and practical advice. Int. J. Mental Health Addict. 7(4), 555–562 (2009)

    Article  Google Scholar 

  30. Kornilova, T.V.: Experimental Psychology. Jurajt, Moscow (2014). (in Russian)

    Google Scholar 

  31. Kuhn, T.S.: The Structure of Scientific Revolutions. International Encyclopedia of United Science, vol. 2. The University of Chicago Press, Chicago (1962)

    Google Scholar 

  32. Lim, C., Jung, H.: A study on the military serious game. Adv. Sci. Technol. Lett. 39, 73–77 (2013). Proceedings. SERSC 2013

    Article  Google Scholar 

  33. Marchand, A., Hennig-Thurau, T.: Value creation in the video game industry: industry economics, consumer benefits, and research opportunities. J. Interact. Market. 27, 141–157 (2013)

    Article  Google Scholar 

  34. Marsden, J.: The essential checklist for making an awesome video game, According to Futurlab (2013). http://indiegames.com/2013/07/the_essential_checklist_for_ma_1.html

  35. Martin, D.W.: Doing Psychology Experiments. Thompson Higher Education, Belmont (2008)

    Google Scholar 

  36. Martins, T., Araújo, M., Carvalho, V., Soares, F., Torrão, L.: PhysioVinci – a first approach on a physical rehabilitation game. In: Ma, M., Oliveira, M.F., Baalsrud Hauge, J. (eds.) SGDA 2014. LNCS, vol. 8778, pp. 1–9. Springer, Heidelberg (2014)

    Google Scholar 

  37. Martin-SanJosé, J.-F., Juan, M.-C., Segui, I., Garcia-Garcia, I.: The effects of computer-based games and collaboration in large groups vs collaboration in pair or traditional methods. Comput. Educ. 87, 42–54 (2015)

    Article  Google Scholar 

  38. Mayer, I., Zhou, Q., Keijser, X., Abspoel, L.: Gaming the future of the ocean: the marine spatial planning challenge 2050. In: Ma, M., Oliveira, M.F., Baalsrud Hauge, J. (eds.) SGDA 2014. LNCS, vol. 8778, pp. 150–162. Springer, Heidelberg (2014)

    Google Scholar 

  39. Omelina, L., Jansen, B., Bonnechère, B., Van Sint Jan, S., Cornelis, J.: Serious games for physical rehabilitation: designing highly configurable and adaptable games. In: Proceeding 9th International Conference on Disability, Virtual Reality and Associated Technologies Laval, France, 10–12 September 2012 (ICDVRAT 2012), pp. 195–201. ICDVRAT (2012)

    Google Scholar 

  40. Operationalization. Wikipedia. https://en.wikipedia.org/wiki/Operationalization

  41. Popper, K.: Normal Science and its Dangers. In: Lakatos, I., Musgrave, A. (eds.) Criticism and the Growth of Knowledge, pp. 51–58. Cambridge University Press, Cambridge (1970)

    Chapter  Google Scholar 

  42. Popper, K.: The Logic of Scientific Discovery. Routledge, London (2002)

    MATH  Google Scholar 

  43. Raulin, M.L., Graziano, A.M.: Quasi-experiments and correlational studies. In: Colman, A.M. (ed.) Companion Encyclopedia of Psychology, vol. 2, pp. 1124–1141. Routledge, London (1994)

    Google Scholar 

  44. Reliability. In: Merriam-Webster.com. Merriam-Webster. http://www.merriam-webster.com/dictionary/reliability

  45. Ricciardi, F., De Paolis, L.T.: A Comprehensive review of serious games in health professions. Int. J. Comput. Games Technol. (2014). http://dx.doi.org/10.1155/2014/787968

  46. Senevirathne, S.G., Kodagoda, M., Kadle, V., Haake, S.J., Senior, T., Heller, B.W.: Application of serious games to sports, health and exercise. In: Proceedings of the 6th SLIIT Research Symposium, Sri Lanka, vol. 4, pp. 6–9 (2011)

    Google Scholar 

  47. Shubert, T., Strobach, T.: Video game experience optimized executive control skills—on false positives and false negatives: reply to boot and simons. Acta Psychol. 141, 278–280 (2012)

    Article  Google Scholar 

  48. Tomé, R.M., Pereira, J.M., Oliveira, M.: Using serious games for cognitive disabilities. In: Ma, M., Oliveira, M.F., Baalsrud Hauge, J. (eds.) SGDA 2014. LNCS, vol. 8778, pp. 34–47. Springer, Heidelberg (2014)

    Google Scholar 

  49. Tong, T., Chignell, M., Tierney, M.C., Masella, C.: A Serious game for clinical assessment of cognitive status: validation study. JMIR Serious Games 4(1), e7 (2016). doi:10.2196/games.5006

    Article  Google Scholar 

  50. Validity. In: Merriam-Webster.com. Merriam-Webster. http://www.merriam-webster.com/dictionary/validity

  51. Van der Spek, E.D.: Experiments in serious game design: a cognitive approach. Utrecht University Repository (Dissertation). Utrecht University, Utrecht (2011)

    Google Scholar 

  52. Van Muijden, J., Band, G.P.H., Hommel, B.: Online games training aging brains: limited transfer to cognitive control functions. Front. Hum. Neurosci. 6, 221 (2012). doi:10.3389/fnhum.2012.00221

    Google Scholar 

  53. Wouters, P., van Nimwegen, C., van Oostendorp, H., van der Spek, E.D.: A meta-analysis of the cognitive and motivational effects of serious games. J. Educ. Psychol. 105, 249 (2013). doi:10.1037/a0031311

    Article  Google Scholar 

  54. Wrzesien, M., Alcañiz, M., Botella, C., Burkhardt, J.-M., Lopez, J.B., Ortega, A.R.: A pilot evaluation of a therapeutic game applied to small animal phobia treatment. In: Ma, M., Oliveira, M.F., Baalsrud Hauge, J. (eds.) SGDA 2014. LNCS, vol. 8778, pp. 10–20. Springer, Heidelberg (2014)

    Google Scholar 

  55. Yoo, K., Ahn, J., Lee, W.: A design of the stress relief game based on autonomic nervous system. In: Park, J.H., Jeong, Y.-S., Park, S.O., Chen, H.-C. (eds.) EMC 2012. LNEE, vol. 181, pp. 371–376. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  56. Zakari, H.M., Ma, M., Simmons, D.: A review of serious games for children with autism spectrum disorders (ASD). In: Ma, M., Oliveira, M.F., Baalsrud Hauge, J. (eds.) SGDA 2014. LNCS, vol. 8778, pp. 93–106. Springer, Heidelberg (2014)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Pedagogics and Medical Psychology Department, Faculty of Higher Nursing Training, Psychology and Social Work, Sechenov First Moscow State Medical University, 8-2 Trubetskaya street, Moscow, 119991, Russia

    Nataliya V. Bogacheva

Authors
  1. Nataliya V. Bogacheva
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nataliya V. Bogacheva .

Editor information

Editors and Affiliations

  1. RheinMain University of Applied Sciences , Wiesbaden, Germany

    Ralf Dörner

  2. Technische Universität Darmstadt , Darmstadt, Germany

    Stefan Göbel

  3. TU Graz , Graz, Austria

    Michael Kickmeier-Rust

  4. University of Duisburg-Essen , Duisburg, Germany

    Maic Masuch

  5. TU Kaiserslautern , Kaiserslautern, Germany

    Katharina Zweig

Further Reading

Further Reading

For more experiments on serious games, see:

  • Van der Spek, E.D.: Experiments in Serious Game Design: a Cognitive Approach. Utrecht University Repository (Dissertation). Utrecht University, Utrecht (2011)

For deeper knowledge about experimental and other research types in social sciences and psychology, as well as for common statistical procedures see:

  • Kantowitz, B.H., Roediger, H.L.III, Elmes, D.G.: Experimental Psychology. Wadsworth, Belmont (2009)

  • Martin D.W.: Doing Psychology Experiments. Thompson Higher Education, Belmont (2008)

  • Goodwin C.J. Research in Psychology: Methods and Design. Wiley, Danvers (2010)

To learn about online experiments’ possibility, see:

  • Reips, U.-D.: Standards for Internet-based Experimenting. In: Experimental Psychology. 49(9), 243–256 (2002)

  • Reips, U.-D., Krantz, J.H.: Conducting True Experiments on the Web. In: Gosling, S., Johnson, J. (eds.) Advanced Methods for Conducting Online Behavioral Research, pp. 193–216. American Psychological Association, Washington DC (2010)

For basic knowledge about latent variables (variables, which can only be discovered through statistical procedures, very common in psychology and social sciences) see:

  • Bollen, K.A.: Latent Variables in Psychology and the Social Sciences. In: Annu. Rev. Psychol. 53, 605–634 (2002)

For advanced knowledge in statistical methods (multivariate analysis of variance, multiple regression, etc.), see:

  • Foster J., Barkus, E., Yavorsky, C.: Understanding and Using Advanced Statistics: A Practical Guide for Students. SAGE Puclications, London, Thousand Oaks, New Delhi (2006)

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this chapter

Cite this chapter

Bogacheva, N.V. (2016). The Experimental Method as an Evaluation Tool in Serious Games Research and Development. In: Dörner, R., Göbel, S., Kickmeier-Rust, M., Masuch, M., Zweig, K. (eds) Entertainment Computing and Serious Games. Lecture Notes in Computer Science(), vol 9970. Springer, Cham. https://doi.org/10.1007/978-3-319-46152-6_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-46152-6_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-46151-9

  • Online ISBN: 978-3-319-46152-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation