Pilot Study of Piano Learning with AR Smart Glasses Considering Both Single and Paired Play

  • Honghu Pan
  • Xingxi HeEmail author
  • Hong Zeng
  • Jia Zhou
  • Sai Tang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10927)


To ease the piano learning process for novices and to foster their interests, a piano learning app operated on the AR smart glasses, Epson Moeverio BT-300, was created considering both single and paired play. The app presents an AR screen in front of the user’s view without blocking the view of surroundings, supplying note information by mapping the keys to be pressed on the AR screen and the keys on a real piano. Therefore no prior knowledge of the sheet music notation is required for users. An evaluation experiment was conducted and 13 volunteers were invited including two elderly people. After the practice mode, the participants entering into the play mode are required to play a single play song and a paired play song. For the paired play mode, one learner plays the right hand melody and the other partner plays the left hand harmony. The best performance in the paired play was generated by a pair of the most familiar friends even though they showed very ordinary performance in their single play among all the participants. The elderly people expressed their special interests in the paired play for they found it more enjoyable with the companionship of the partner. The perceived enjoyment is influenced by many factors such as the degree of comfort of wearing the smart glasses, the perceived enjoyment of the melody of the songs, mapping process, the degree of practice, spatial ability, other personal experiences, etc.


Piano learning AR smart glasses Single play and paired play 


  1. 1.
    Rogers, K., Röhlig, A., Weing, M., Gugenheimer, J., Könings, B., Klepsch, M., Weber, M.: Piano: faster piano learning with interactive projection. In: Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces, pp. 149–158. ACM, November 2014Google Scholar
  2. 2.
    Huang, F., Zhou, Y., Yu, Y., Wang, Z., Du, S.: Piano AR: a markerless augmented reality based piano teaching system. In: 2011 International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC), vol. 2, pp. 47–52. IEEE, August 2011Google Scholar
  3. 3.
    Chow, J., Feng, H., Amor, R., Wünsche, B.C.: Music education using augmented reality with a head mounted display. In: Proceedings of the Fourteenth Australasian User Interface Conference-Volume 139, pp. 73–79. Australian Computer Society Inc., January 2013Google Scholar
  4. 4.
    Das, S., Glickman, S., Hsiao, F.Y., Lee, B.: Music Everywhere–Augmented Reality Piano Improvisation Learning System (2017)Google Scholar
  5. 5.
    Castro, V.C.D., Carreira, L.: Leisure activities and attitude of institutionalized elderly people: a basis for nursing practice. Rev. latino-am. enferm. 23(2), 307–314 (2015)CrossRefGoogle Scholar
  6. 6.
    Kim, J., Yamada, N., Heo, J., Han, A.: Health benefits of serious involvement in leisure activities among older Korean adults. Int. J. Qual. Stud. Health Well-Being 9(1), 24616 (2014)CrossRefGoogle Scholar
  7. 7.
    Uptampohtiwat, T.: 1 to 5 piano: musical medicine. J. Urban Cult. Res. 6, 50–57 (2013)Google Scholar
  8. 8.
    Albert, W., Dixon, E.: Is this what you expected? the use of expectation measures in usability testing. In: Proceedings of the Usability Professionals Association 2003 Conference, Scottsdale, AZ, June 2003Google Scholar
  9. 9.
    Sun, H., Zhang, P.: Causal relationships between perceived enjoyment and perceived ease of use: an alternative approach. J. Assoc. Inf. Syst. 7(9), 24 (2006)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Honghu Pan
    • 1
  • Xingxi He
    • 1
    Email author
  • Hong Zeng
    • 1
  • Jia Zhou
    • 2
  • Sai Tang
    • 1
  1. 1.The State Key Laboratory of Mechanical TransmissionsChongqing UniversityChongqingChina
  2. 2.Department of Industrial EngineeringChongqing UniversityChongqingChina

Personalised recommendations