Abstract
This paper presents a novel game design that allows players to learn how to cast magic spells that combine hand gestures and speech. This game uses the imperfect recognition performance in speech and gesture recognition systems to its advantage to make the game challenging and interesting. Our game uses a Wii remote encased in a wand and a microphone to track player’s gestures and speech which are then recognized to determine if they have performed the spell correctly. Visual feedback then provides confirmation of success. Through the game, players learn to adjust their speaking and movement patterns in order to meet the requirements of the recognition systems. This effectively mimics the characteristics of casting spells correctly such that players are trying to adjust their performance so that an “oracle” recognizes their speech and movement to have a magical outcome. A user study has confirmed the validity of the idea and establishes the accuracy required to create an interesting game based on the theory of channels of flow.
Chapter PDF
Similar content being viewed by others
References
Pinelle, D., Wang, N., Stach, T.: Heuristic Evaluation for Games: Usability Principles for Video Game Design. In: CHI 2008, vol. 5, pp. 1453–1462 (April 2008)
Rowling, J.K.: Harry Potter and the Philosopher’s stone (1997)
Rowling, J.K.: Harry Potter and the Goblet of Fire (2000)
Lecture 12: An overview of Speech Recognition, www.cs.rochester.edu/u/james/CSC248/Lec12.pdf (retrieved February 25, 2010)
Kiili, K.: Content creation challenges and flow experience in educational games: The IT-Emperor case. The Internet and Higher Education 8(3), 183–198 (2005)
http://www.xbox.com/en-US/kinect/default.htm (retrieved June 18, 2010)
http://en.wikipedia.org/wiki/Lego_Harry_Potter (retrieved March 12, 2010)
http://www.ea.com/games/harry-potter-half-blood-prince (retrieved March 12, 2010)
Buxton, B.: Chapter 14: Gesture Based Interaction (May 2009), http://www.billbuxton.com/input14.Gesture.pdf (Retrieved March 6, 2010)
Leong, T., Lai, J., Pong, P., et al.: Wii Want to Write: An Accelerometer Based Gesture Recognition System. In: International Conference on Recent and Emerging Advanced Technologies in Engineering 2009, Malaysia (2009)
Mlích, J.: Wiimote Gesture Recognition. In: Proceedings of the 15th Conference and Competition STUDENT EEICT 2009, vol. 4, pp. 344–349 (2009)
Q6.1: What is speech recognition?, http://www.speech.cs.cmu.edu/comp.speech/Section6/Q6.1.html (retrieved February 25, 2010)
Walker, W., et al.: Sphinx-4: A flexible open source framework for speech recognition (2004), http://research.sun.com/techrep/2004/smli_tr-2004-139.pdf
Huang, X., Alleva, F., Hon, H.W., Hwang, M.Y., Rosenfeld, R.: The SPHINX-II speech recognition system: an overview. Computer Speech and Language 7(2), 137–148 (1993)
http://wiiyourself.gl.tter.org/ (retrieved March 16, 2010)
http://cmusphinx.sourceforge.net/sphinx4/ (retrieved March 18, 2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wang, C., Liu, Z., Fels, S. (2010). Everyone Can Do Magic: An Interactive Game with Speech and Gesture Recognition. In: Yang, H.S., Malaka, R., Hoshino, J., Han, J.H. (eds) Entertainment Computing - ICEC 2010. ICEC 2010. Lecture Notes in Computer Science, vol 6243. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15399-0_4
Download citation
DOI: https://doi.org/10.1007/978-3-642-15399-0_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15398-3
Online ISBN: 978-3-642-15399-0
eBook Packages: Computer ScienceComputer Science (R0)