Multimedia Tools and Applications

, Volume 78, Issue 10, pp 13713–13730 | Cite as

Playing to play: a piano-based user interface for music education video-games

  • Edoardo MicheloniEmail author
  • Marco Tramarin
  • Antonio Rodà
  • Federico Chiaravalli


This paper presents and discusses a case-study where a video-game is the means through which the user is induced to learn to use a complex and not-intuitive control interface such as the keyboard of a piano. The interaction paradigm is based on the idea, common to the Tangible Interfaces, to employ everyday objects, or anyway not designed for video games, as control user interface. In this research, the case study is the video game called Musa, in which players are led into an imaginary world where music is magic and every action is carried out through it. Combined with an educational path, this is thought to make players able to learn and, in a successive moment, to abstract this knowledge from the game, just by playing. Musa was evaluated in an experimental setup with 51 kids between 6 and 11 years old. Results show that subjects are able to improve their knowledge of the piano keyboard after two sessions of game and no significant differences were found between children with pre-acquired knowledge of the keyboard and the others.


Tangible interfaces Videogame Piano-based Music education Acoustic stimuli 



This research has been supported by Musa s.r.l.


  1. 1.
    Alty JL “Can we use music in computer-human communication?” BCS HCI. 1995Google Scholar
  2. 2.
    Avanzini F et al. “Archaeology and virtual acoustics - a pan flute from ancient Egypt.” Proceedings of the 12th International Conference on Sound and Music Computing (SMC 2015). 2015Google Scholar
  3. 3.
    Avanzini F, et al. “Virtual reconstruction of an ancient greek pan flute.” In Proc. Int. Conf. Sound and Music Computing (SMC 2016). 2016Google Scholar
  4. 4.
    Bressan F, Canazza S (2014) The challenge of preserving interactive sound art: a multi-level approach. Int J Arts Technol 7(4):294–315CrossRefGoogle Scholar
  5. 5.
    Canazza S, Fantozzi C, Pretto N. “Accessing tape music documents on mobile devices.” ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 12.1s 2015Google Scholar
  6. 6.
    Caroux L, Isbister K, Le Bigot L, Vibert N (2015) Player–video game interaction: a systematic review of current concepts. Comput Hum Behav 48:366–381CrossRefGoogle Scholar
  7. 7.
    De Cheveigné A, Kawahara H (2002) YIN, a fundamental frequency estimator for speech and music. J Acoust Soc Am 111.4:1917–1930CrossRefGoogle Scholar
  8. 8.
    Deterding S, Dixon D, Khaled R, Nacke L “From game design elements to gamefulness: defining gamification.” Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments. ACM, 2011Google Scholar
  9. 9.
    Ephraim Y, Malah D (1984) Speech enhancement using a minimum-mean square error short-time spectral amplitude estimator. IEEE Trans Acoust Speech Signal Process 32(6):1109–1121CrossRefGoogle Scholar
  10. 10.
    Fantozzi C, Bressan F, Pretto N, Canazza S (2017) Tape music archives: from preservation to access. Int J Digit Libr 18(3):233–249CrossRefGoogle Scholar
  11. 11.
    Gaver WW (1986) Auditory icons: using sound in computer interfaces. Human-Comput Int 2(2):167–177CrossRefGoogle Scholar
  12. 12.
    Gerkmann T, Hendriks RC (2012) Unbiased MMSE-based noise power estimation with low complexity and low tracking delay. IEEE Trans Audio Speech Lang Process 20(4):1383–1393CrossRefGoogle Scholar
  13. 13.
    Hanna L, Risden K, Alexander K (1997) Guidelines for usability testing with children. Interactions 4(5):9–14CrossRefGoogle Scholar
  14. 14.
    Ishii H. “Tangible bits: beyond pixels.” Proceedings of the 2nd international conference on Tangible and embedded interaction. ACM, 2008Google Scholar
  15. 15.
    Jacob RJK, Audrey Girouard LM Hirshfield MS Horn OS, Erin Treacy Solovey and Jamie Zigelbaum "Reality-based interaction: a framework for post-WIMP interfaces." Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 2008Google Scholar
  16. 16.
    Luc N, Lesaffre M, Leman M. “The musical instrument as a natural extension of the musician.” the 5th Conference of Interdisciplinary Musicology. LAM-Institut jean Le Rond d'Alembert, 2009Google Scholar
  17. 17.
    Magerkurth C, Cheok AD, Mandryk RL, Nilsen T (2005) Pervasive games: bringing computer entertainment back to the real world. Comput Entertain (CIE) 3(3):4–4CrossRefGoogle Scholar
  18. 18.
    Markopoulos P, Bekker M (2003) On the assessment of usability testing methods for children. Interact Comput 15(2):227–243CrossRefGoogle Scholar
  19. 19.
    Orio N, Lemouton S, Schwarz D. “Score following: State of the art and new developments.” Proceedings of the 2003 conference on New interfaces for musical expression. National University of Singapore, 2003Google Scholar
  20. 20.
    Poole A, Ball LJ (2006) Eye tracking in HCI and usability research. Encyclop Human Comput Int 1:211–219CrossRefGoogle Scholar
  21. 21.
    Roussou M (2004) Learning by doing and learning through play: an exploration of interactivity in virtual environments for children. Comput Entertain (CIE) 2(1):10–10CrossRefGoogle Scholar
  22. 22.
    Schank RC, Tamara R (1999) Berman, and Kimberli a. Macpherson. Learning by doing. Inst-Design Theor Models: A New Paradigm Inst Theory 2:161–181Google Scholar
  23. 23.
    Sigtia S, Benetos E, Dixon S An end-to-end neural network for polyphonic piano music transcription. IEEE/ACM Trans Audio, Speech Lang Proc (TASLP) 24.5(2016):927–939Google Scholar
  24. 24.
    Sohn J, Kim NS, Sung W (1999) A statistical model-based voice activity detection. IEEE Sign Proc Lett 6(1):1–3CrossRefGoogle Scholar
  25. 25.
    Wanderley MM, Orio N (2002) Evaluation of input devices for musical expression: borrowing tools from hci. Comput Music J 26(3):62–76CrossRefGoogle Scholar
  26. 26.
    Wigdor D, Wixon D. “Brave NUI world: designing natural user interfaces for touch and gesture”. Elsevier, 2011Google Scholar
  27. 27.
    Young GW, Murphy D. “Hci models for digital musical instruments: Methodologies for rigorous testing of digital musical instruments.” International Symposium on Computer Music Multidisciplinary Research (CMMR). 2015Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.University of PadovaPadovaItaly
  2. 2.Musa s.r.l.MilanItaly

Personalised recommendations