Abstract
Virtual musical instruments (VMIs) are Augmented Reality (AR) or Virtual Reality (VR) based applications that focus on the interaction with real or virtual musical objects through AR or VR mediums. For such purpose, different interaction tools and techniques such as fiducial markers, RGB and depth cameras, VR and AR headsets are used. Though modern AR and VR devices provide better interaction interfaces, they are more expensive than traditional ones that are unfortunately not robust as the modern devices. To provide a robust and affordable AR and VR interaction interface for real-time applications, we investigate the use of affordable body-worn inertial measurement unit (IMU), a web camera, an RGB light as a simple, cheaper and reliable solution for the interaction with virtual percussion instruments. The performance of the proposed system was evaluated through a series of feedback from fifteen users. In general, the framework was considered as intuitive and tangible.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Poupyrev, I., Berry, R., Kurumisawa, J., Nakao, K., Billinghurst, M., Airola, C., Kato, H., Yonezawa, T., Baldwin, L.: Augmented groove: collaborative jamming in augmented reality. In: ACM SIGGRAPH 2000 Conference Abstracts and Applications, p. 77 (2000)
Poupyrev, I., Berry, R., Billinghurst, M., Kato, H., Nakao, K., Baldwin, L., Kurumisawa, J.: Augmented reality interface for electronic music performance. In: Proceedings of HCI, pp. 805–808 (2001)
Jordà , S.: Interactive music systems for everyone: exploring visual feedback as a way for creating more intuitive, efficient and learnable instruments. In: Proceedings of the Stockholm Music Acoustics Conference, pp. 6–9, August 2003
Karjalainen, M., Maki-Patola, T.: Physics-based modeling of musical instruments for interactive virtual reality. In: 2004 IEEE 6th Workshop on Multimedia Signal Processing, pp. 223–226. IEEE (2004)
Wöldecke, B., Marinos, D., Pogscheba, P., Geiger, C., Herder, J., Schwirten, T.: RadarTHEREMIN-Creating musical expressions in a virtual studio environment. In: 2011 IEEE International Symposium on VR Innovation (ISVRI), pp. 345–346. IEEE (2011)
Olivieri, D., Conde, I.G., Vila-Sobrino, X.A.: AR-based virtual musical instruments using SMC tracking. In: 2011 6th Iberian Conference on Information Systems and Technologies (CISTI), pp. 1–7. IEEE (2011)
Rastogi, A., Joshi, A.: Virtual Musical Instruments (2015)
Lobo, N.: V-Drum: an augmented reality drum kit. Int. J. Adv. Res. Comput. Commun. Eng. 4(10), October 2015
Yamabe, T., Asuma, H., Kiyono, S., Nakajima, T.: Feedback design in augmented musical instruments: a case study with an ar drum kit. In: 2011 IEEE 17th International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA), vol. 2, pp. 126–129. IEEE (2011)
Nilsson, S., Vechev, V., Yeh, A., Hedler, C.: Holo beats: design and development of an AR system to teach drums. In: Proceedings of SIDeR 2016, The 12th Student Interaction Design Research Conference, Malmö (2016)
Fernandez, C.A.T., Paliyawan, P., Yin, C.C., Thawonmas, R.: Piano learning application with feedback provided by an AR virtual character. In: 2016 IEEE 5th Global Conference on Consumer Electronics, pp. 1–2. IEEE (2016)
Cabral, M., Montes, A., Roque, G., Belloc, O., Nagamura, M., Faria, R.R.A., Teubl, F., Kurashima, C., Lopes, R., Zuffo, M.: Crosscale: a 3D virtual musical instrument interface. In: 2015 IEEE Symposium on 3D User Interfaces (3DUI), pp. 199–200. IEEE (2015)
Fillwalk, J.: Chromachord: a virtual musical instrument. In: 2015 IEEE Symposium on 3D User Interfaces (3DUI), pp. 201–202. IEEE (2015)
Moore, A.G., Howell, M.J., Stiles, A.W., Herrera, N.S., McMahan, R.P.: Wedge: a musical interface for building and playing composition-appropriate immersive environments. In: 2015 IEEE Symposium on 3D User Interfaces (3DUI), pp. 205–206. IEEE (2015)
Hariadi, R.R., Kuswardayan, I.: Design and implementation of Virtual Indonesian Musical Instrument (VIMi) application using Leap Motion Controller. In: 2016 International Conference on Information & Communication Technology and Systems (ICTS), pp. 43–48. IEEE (2016)
Hsu, M.-H., Kumara, W.G.C.W., Shih, T.K., Cheng, Z.: Spider King: virtual musical instruments based on microsoft kinect. In: 2013 International Joint Conference on Awareness Science and Technology and Ubi-Media Computing (iCAST-UMEDIA), pp. 707–713. IEEE (2013)
Burks, N., Smith, L., Saquer, J.: A virtual xylophone for music education. In: 2016 IEEE International Symposium on Multimedia (ISM), pp. 409–410. IEEE (2016)
Correa, A.G.D., Ficheman, I.K., do Nascimento, M., de Deus Lopes, R.: Computer assisted music therapy: a case study of an augmented reality musical system for children with cerebral palsy rehabilitation. In: Ninth IEEE International Conference on Advanced Learning Technologies, ICALT 2009, pp. 218–220. IEEE (2009)
Chouvatut, V., Jindaluang, W.: Virtual piano with real-time interaction using automatic marker detection. In: 2013 International Computer Science and Engineering Conference (ICSEC), pp. 222–226. IEEE (2013)
MbientLab. https://mbientlab.com/product/metamotionr/
MbientLab. https://mbientlab.com/cppdocs/latest/dataprocessor.html#pulse
Guitar Center. http://www.guitarcenter.com/Sound-Percussion-Labs/Bell-Kit-w-Rolling-Cart-2-1-2-OCTAVE-1385392679518.gc?pfm=item_page.rrt1|ClickCP
Taobao. https://item.taobao.com/item.htm?spm=a1z09.2.0.0.34792e8dG0R4A6&id=558971532837&_u=c8icdh45a36
Acknowledgments
The research was sponsored by CAS-TWAS President’s Fellowship.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Fangbemi, A.S., Zhang, Y. (2018). Wrist-Worn Sensor-Based Tangible Interface for Virtual Percussion Instruments. In: De Paolis, L., Bourdot, P. (eds) Augmented Reality, Virtual Reality, and Computer Graphics. AVR 2018. Lecture Notes in Computer Science(), vol 10850. Springer, Cham. https://doi.org/10.1007/978-3-319-95270-3_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-95270-3_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-95269-7
Online ISBN: 978-3-319-95270-3
eBook Packages: Computer ScienceComputer Science (R0)