Skip to main content
SpringerLink
Log in
Book cover

International Conference on Innovative Technologies and Learning

ICITL 2019: Innovative Technologies and Learning pp 194–203Cite as

Enhanced Learning of Jazz Chords with a Projector Based Piano Keyboard Augmentation

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11937))

Abstract

Learning jazz piano is considered technically difficult. Most people cannot afford private piano tuition and there are many freely available video tutorials on the Internet. This study identified a set of challenging topics associated with learning jazz piano based on popular YouTube jazz piano tutorials. The results suggest that learners are struggling to understand the construction of complex chords in the 12 keys. Most tutorial videos show two parallel keyboards, one physical with the real hands and one virtual keyboard, which the viewers must map onto their own keyboard. Based on these results a series of visualization approaches were explored through a low-fi prototype by augmenting the physical piano keyboard with voicing instructions using a projector.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Serafin, S., Adjorlu, A., Nilsson, N., Thomsen, L., Nordahl, R.: Considerations on the use of virtual and augmented reality technologies in music education. In: 2017 IEEE Virtual Reality Workshop on K-12 Embodied Learning through Virtual & Augmented Reality (KELVAR). IEEE (2017)

    Google Scholar 

  2. Sandnes, F.E., Jian, H.L.: Sketching with Chinese calligraphy. Interactions 19(6), 62–66 (2012)

    Article  Google Scholar 

  3. Sandnes, F.E., Jian, H.-L.: Pair-wise variability index: evaluating the cognitive difficulty of using mobile text entry systems. In: Brewster, S., Dunlop, M. (eds.) Mobile HCI 2004. LNCS, vol. 3160, pp. 347–350. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-28637-0_35

    Chapter  Google Scholar 

  4. Zhang, Y., Liu, S., Tao, L., Yu, C., Shi, Y., Xu, Y.: ChinAR: facilitating Chinese Guqin learning through interactive projected augmentation. In: Proceedings of Third International Symposium of Chinese CHI, pp. 23–31. ACM (2015)

    Google Scholar 

  5. McCarthy, K.M., McVey, J.M., Ericson, J.M., Eveland, Z.E., Taylor, M., Ruscher, T.H.: U.S. patent no. 9,092,994. U.S. Patent and Trademark Office, Washington, DC (2015)

    Google Scholar 

  6. Sandnes, F.E., Eika, E.: Head-mounted augmented reality displays on the cheap: a DIY approach to sketching and prototyping low-vision assistive technologies. In: Antona, M., Stephanidis, C. (eds.) UAHCI 2017. LNCS, vol. 10278, pp. 167–186. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58703-5_13

    Chapter  Google Scholar 

  7. Trujano, F., Khan, M., Maes, P.: ARPiano efficient music learning using augmented reality. In: Wu, T.-T., Huang, Y.-M., Shadieva, R., Lin, L., Starčič, A.I. (eds.) ICITL 2018. LNCS, vol. 11003, pp. 3–17. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99737-7_1

    Chapter  Google Scholar 

  8. Hackl, D., Anthes, C.: HoloKeys-an augmented reality application for learning the Piano. In: Forum Media Technology, pp. 140–144. (2017)

    Google Scholar 

  9. Zeng, H., He, X., Pan, H.: FunPianoAR: a novel AR application for piano learning considering paired play based on multi-marker tracking. In: Journal of Physics: Conference Series 1229, no. 1, p. 012072. IOP Publishing (2019)

    Google Scholar 

  10. Birhanu, A., Rank, S.: KeynVision: exploring piano pedagogy in mixed reality. In: Extended Abstracts Publication of the Annual Symposium on Computer-Human Interaction in Play, pp. 299–304. ACM (2017)

    Google Scholar 

  11. Liang, H., et al.: Barehanded music: real-time hand interaction for virtual piano. In: Proceedings of 20th ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, pp. 87–94. ACM (2016)

    Google Scholar 

  12. Rogers, K., et al.: Piano: faster piano learning with interactive projection. In: Proceedings of Ninth ACM International Conference on Interactive Tabletops and Surfaces, pp. 149–158. ACM (2014)

    Google Scholar 

  13. Weing, M., et al.: PIANO: enhancing instrument learning via interactive projected augmentation. In: Proceedings of ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication, pp. 75–78. ACM (2013)

    Google Scholar 

  14. Raymaekers, L., Vermeulen, J., Luyten, K., Coninx, K.: Game of tones: Learning to play songs on a piano using projected instructions and games. In: CHI’14 Extended Abstracts on Human Factors in Computing Systems, pp. 411–414. ACM (2014)

    Google Scholar 

  15. Chiang, P.Y., Sun, C.H.: Oncall piano sensei: Portable ar piano training system. In: Proceedings of 3rd ACM Symposium on Spatial User Interaction, p. 134. ACM (2015)

    Google Scholar 

  16. Sun, C.H., Chiang, P.Y.: Mr. piano: a portable piano tutoring system. In: 2018 IEEE XXV International Conference on Electronics, Electrical Engineering and Computing (INTERCON). IEEE (2018)

    Google Scholar 

  17. Gerry, L., Dahl, S., Serafin, S.: ADEPT: exploring the design, pedagogy, and analysis of a mixed reality application for piano training. In: Proceedings of 16th Sound & Music Computing Conference (2019)

    Google Scholar 

  18. Xiao, X., Aguilera, P., Williams, J., Ishii, H.: MirrorFugue iii: conjuring the recorded pianist. In: CHI Extended Abstracts, pp. 2891–2892. ACM (2013)

    Google Scholar 

  19. Xiao, X., Ishii, H.: MirrorFugue: communicating hand gesture in remote piano collaboration. In: Proceedings of Fifth International Conference on Tangible, Embedded, and Embodied Interaction, pp. 13–20. ACM (2011)

    Google Scholar 

  20. Xiao, X., Ishii, H.: Inspect, embody, invent: a design framework for music learning and beyond. In: Proceedings of 2016 CHI Conference on Human Factors in Computing Systems, pp. 5397–5408. ACM (2016)

    Google Scholar 

  21. Suzuki, G., Murase, T., Fujii, Y.: Projecting recorded expert hands at real size, at real speed, and onto real objects for manual work. In: Companion Publication of the 21st International Conference on Intelligent User Interfaces, pp. 13–17. ACM (2016)

    Google Scholar 

  22. Nemoto, T., Umezu, N.: Support for keyboard performing practice with mirrored video of model performances. In: 2018 IEEE 7th Global Conference on Consumer Electronics (GCCE), pp. 90–93. IEEE (2018)

    Google Scholar 

  23. Xiao, X., Puentes, P., Ackermann, E., Ishii, H.: Andantino: teaching children piano with projected animated characters. In: Proceedings of 15th International Conference on Interaction Design and Children, pp. 37–45. ACM (2016)

    Google Scholar 

  24. Hsiao, C.P., Li, R., Yan, X., Do, E.Y.L.: Tactile teacher: Sensing finger tapping in piano playing. In: Proceedings Ninth International Conference on Tangible, Embedded, and Embodied Interaction, pp. 257–260. ACM (2015)

    Google Scholar 

  25. Fujii, K., Russo, S.S., Maes, P., Rekimoto, J.: MoveMe: 3D haptic support for a musical instrument. In: Proceedings of 12th International Conference on Advances in Computer Entertainment Technology. ACM (2015)

    Google Scholar 

  26. Kallionpää, M., Gasselseder, H.P.: Creating a super instrument: composers and pianists reaching beyond their technical and expressive capabilities. In: Proceedings of Audio Mostly 2015 on Interaction with Sound ACM (2015)

    Google Scholar 

  27. Sandnes, F.E., Eika, E.: Visual augmentation of printed materials with intelligent see-through glass displays: a prototype based on smartphone and pepper’s ghost. In: 2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR), pp. 267–273. IEEE (2018)

    Google Scholar 

  28. Rusiñol, M., Chazalon, J., Diaz-Chito, K.: Augmented songbook: an augmented reality educational application for raising music awareness. Multimed. Tools Appl. 77(11), 13773–13798 (2018)

    Article  Google Scholar 

  29. Donahue, C., Simon, I., Dieleman, S.: Piano genie. arXiv preprint arXiv:1810.05246. (2018)

  30. Dias, R., Guedes, C.: GimmeDaBlues app for iOS: overview and ongoing developments. In: SIGGRAPH Asia 2012 Symposium on Apps. ACM (2012)

    Google Scholar 

  31. Snydal, J., Hearst, M.: ImproViz: visual explorations of jazz improvisations. In: CHI 2005 Extended Abstracts on Human Factors in Computing Systems, pp. 1805–1808. ACM (2005)

    Google Scholar 

  32. Ciuha, P., Klemenc, B., Solina, F.: Visualization of concurrent tones in music with colours. In: Proceedings of 18th ACM International Conference on Multimedia, pp. 1677–1680. ACM (2010)

    Google Scholar 

  33. Bergstrom, T., Karahalios, K., Hart, J.C.: Isochords: visualizing structure in music. In: Proceedings of Graphics Interface 2007, pp. 297–304. ACM (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Oslo Metropolitan University, Oslo, Norway

    Frode Eika Sandnes & Evelyn Eika

  2. Kristiania University College, Oslo, Norway

    Frode Eika Sandnes

Authors
  1. Frode Eika Sandnes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Evelyn Eika
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Frode Eika Sandnes .

Editor information

Editors and Affiliations

  1. The Arctic University of Norway, Tromsø, Norway

    Lisbet Rønningsbakk

  2. National Yunlin University of Science and Technology, Douliu, Taiwan

    Ting-Ting Wu

  3. Oslo Metropolitan University, Oslo, Norway

    Frode Eika Sandnes

  4. National Cheng Kung University, Tainan City, Taiwan

    Yueh-Min Huang

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sandnes, F.E., Eika, E. (2019). Enhanced Learning of Jazz Chords with a Projector Based Piano Keyboard Augmentation. In: Rønningsbakk, L., Wu, TT., Sandnes, F., Huang, YM. (eds) Innovative Technologies and Learning. ICITL 2019. Lecture Notes in Computer Science(), vol 11937. Springer, Cham. https://doi.org/10.1007/978-3-030-35343-8_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-35343-8_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-35342-1

  • Online ISBN: 978-3-030-35343-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation