Listen and Play: Auditory-Motor Interaction in a Bard’s Tale Video Game

  • Katya A. Alvarez-MolinaEmail author
  • Aaron Jöllenbeck
  • Rainer Malaka
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 840)


Serious video games represent an immersive way of learning, providing an interaction between learning and game-play. Serious games have shown to be motivational and train the player to acquire certain skills. However, designers should find a balance between pedagogical goals and game-mechanics. Recent investigations show the learning benefits that players obtain by playing music-video games and how these games can support players to improve their musical skills due to embodied music cognition. In the current study, we develop a music-video game to improve the pitch recognition of players. We use the LM-GM model to assess the design of the video game. Moreover, a pilot case study was conducted where the participants performed the video game and answered a Game Experience Questionnaire. Thus, we analyzed the player experience and preliminary results display a relation between learning and game-play. Nevertheless, more studies are needed for appraising the learning effects.


Music learning Serious video game Pitch recognition 



The authors would like to thank Prof. Eva Verena Schmid, professor of music-pedagogy, for her ideas to improve the game design. First author thanks DAAD for funding her doctoral project. Finally, we also would like to thank Grupo Escolar Simón Bolivar from Xalapa, Mexico for providing the participants of the pilot case study.


  1. 1.
    Juul, J.: A Casual Revolution: Reinventing Video Games and Their Players. The MIT Press, Cambridge (2012)Google Scholar
  2. 2.
    Egenfeldt-Nielsen, S.: What makes a good learning game?: Going beyond edutainment. eLearn Mag. 2, Article No. 2 (2011).
  3. 3.
    Prensky, M.: Digital Game-Based Learning. McGraw-Hill, New York (2001)Google Scholar
  4. 4.
    Arnab, S., Lim, T., Carvalho, M. B., Bellotti, F., de Freitas, S., Louchart, S., Suttie, N., Berta, R., De Gloria, A.: Mapping learning and game mechanics for serious games analysis. Br. J. Educ. Technol. 46, 391–411 (2015). Scholar
  5. 5.
    Malaka, R.: How computer games can improve your health and fitness. In: Göbel, S., Wiemeyer, J. (eds.) GameDays 2014. LNCS, vol. 8395, pp. 1–7. Springer, Cham (2014). Scholar
  6. 6.
    de Castell, S., Jenson, J., Thumlert, K.: From simulation to imitation: controllers, corporeality, and mimetic play. Simul. Gaming 45(3), 332–355 (2014). Scholar
  7. 7.
    Juul, J.: The game, the player, the world: looking for a heart of gameness. In: Proceedings of the 2003 DiGRA International Conference: Level Up, pp. 30–45. Utrecht University (2003)Google Scholar
  8. 8.
    Gajadhar, B.J., de Kort, Y.A.W., IJsselsteijn, W.A.: Shared fun is doubled fun: player enjoyment as a function of social setting. In: Markopoulos, P., de Ruyter, B., IJsselsteijn, W., Rowland, D. (eds.) Fun and Games 2008. LNCS, vol. 5294, pp. 106–117. Springer, Heidelberg (2008). Scholar
  9. 9.
    Csikszentmihalyi, M.: Flow: The Psychology of Optimal Experience. Harper Perennial, New York (2008)Google Scholar
  10. 10.
    Hamari, J., Shernoff, D.J., Rowe, E., Coller, B., Asbell-Clarke, J., Edwards, T.: Challenging games help students learn: an empirical study on engagement, flow and immersion in game-based learning. Comput. Hum. Behav. 54(C), 170–179 (2016). Scholar
  11. 11.
    Nijs, L., Coussement, P., Moens, B., Amelick, D., Lesaffre, M., Leman, M.: Interacting with the music paint machine: relating the constructs of flow, experience and presence. Interact. Comput. 24(4), 237–250 (2012). Scholar
  12. 12.
    Alexandrovsky, D., Alvarez, K., Walther-Franks, B., Wollersheim, J., Malaka, R.: Jump‘n’rhythm: a video game for training timing skills. In: Weyers, B., Dittmar, A. (eds.) Mensch und Computer 2016 Workshopband. Gesellschaft für Informatik e.V. Achen, Germany (2016).
  13. 13.
    Leman, M.: Embodied Music Cognition and Mediation Technology. MIT Press, Cambridge (2007)Google Scholar
  14. 14.
    Peppler, K., Downton, M., Lindsay, E., Hay, K.: The Nirvana effect: tapping video games to mediate music learning and interest. Int. J. Learn. Media 3(1), 41–59 (2011). Scholar
  15. 15.
    Lilla, D., Herrlich, M., Malaka, R., Krannich, D.: The influence of music on player performance in exergames for Parkinson’s Patients. In: Herrlich, M., Malaka, R., Masuch, M. (eds.) ICEC 2012. LNCS, vol. 7522, pp. 433–436. Springer, Heidelberg (2012). Scholar
  16. 16.
    Lahav, A., Saltzman, E., Schlaug, G.: Action representation of sound: audiomotor recognition network while listening to newly acquired actions. J. Neurosci. 27(2), 308–314 (2007). Scholar
  17. 17.
    Bangert, M., Haeusler, U., Altenmüller, E.: On practice: how the brain connects piano keys and piano sounds. Ann. N. Y. Acad. Sci. 930, 425–428 (2001). Scholar
  18. 18.
    Lahav, A., Boulanger, A., Schlaug, G., Saltzman, E.: The power of listening: auditory-motor interactions in musical training. Ann. N. Y. Acad. Sci. 1060(1), 189–194 (2005). Scholar
  19. 19.
    Prieto, R., Medina-Medina, N., Paderewski-Rodriguez, P., Gutierrez-Vela, F.L.: Design methodology for educational games based on interactive screenplays. In: Proceedings 2st Congreso de la Sociedad Española para las Ciencias del Videojuego CoSECiVi 2015. CEUR-WS (1394), Barcelona (2015)Google Scholar
  20. 20.
    Carey, R.: Game design canvas. In: Serious Play Conference, Pittsburgh, PA (2015)Google Scholar
  21. 21.
    IJsselsteijn, W., Poels, K., de Kort, Y.A.W.: The game experience questionnaire: development of a self-report measure to assess player experiences of digital games (n.d.)Google Scholar
  22. 22.
    Poels, K., de Kort, Y., Ijsselsteijn, W.: “It is always a lot of fun!”: exploring dimensions of digital game experience using focus group methodology. In: Proceedings of the 2007 Conference on Future Play, pp. 83–184. ACM, Toronto (2007).
  23. 23.
    Schiefele, U.: The role of interest in motivation and learning. In: Collins, J.M., Messick, S. (eds.) Intelligence and Personality: Bridging the Gap in Theory and Measurement, pp. 163–194. Lawrence Erlbaum Associates, Mahwah, N.J. (2001)Google Scholar
  24. 24.
    Cheng, M., She, H., Annetta, L.: Game immersion experience: its hierarchical structure and impact on game-based science learning. J. Comput. Assist. Learn. 31, 232–253 (2015). Scholar
  25. 25.
    Fullagar, C.J., Knight, P.A., Sovern, H.S.: Challenge/skill balance, flow, and performance anxiety. Appl. Psychol.: Int. Rev. 62, 236–259 (2013). Scholar
  26. 26.
    Nacke, L., Lindley, C.: Affective ludology, flow and immersion in a first- person shooter: measurement of player experience. Loading... J. Can. Game Stud. Assoc. 1(5), 1–21 (2009)Google Scholar
  27. 27.
    Baker, R.S.J.D., D’Mello, S.K., Rodrigo, M.M.T., Graesser, A.C.: Better to be frustrated than bored: the incidence, persistence, and impact of learners cognitive affective states during interactions with three different computer-based learning environments. Int. J. Hum.-Comput. Stud. 68(4), 223–241 (2010). Scholar
  28. 28.
    Smeddinck, J.D., Herrlich, M., Malaka, R.: Exergames for physiotherapy and rehabilitation: a medium-term situated study of motivational aspects and impact on functional reach. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI 2015, pp. 4143–4146. ACM, New York (2015).
  29. 29.
    Jäncke, L.: From cognition to action. In: Altenmüler, E., Wiesendanger, M., Kesselring, J. (eds.) Motor, Control and the Brain. Oxford University Press (2006). Scholar
  30. 30.
    Ramos-Mejia, C.M.: La dinmica del violinista. Ricordi Americana (1947)Google Scholar
  31. 31.
    Suzuki, S.: Suzuki Violin School. Violin Part, vol. 1. Summy-Birchard Inc. (Warner), Los Angeles (1978)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Katya A. Alvarez-Molina
    • 1
    Email author
  • Aaron Jöllenbeck
    • 1
  • Rainer Malaka
    • 1
  1. 1.Digital Media Lab, TZIUniversity of BremenBremenGermany

Personalised recommendations