Skip to main content
SpringerLink
Log in
Book cover

Intelligent Video Event Analysis and Understanding pp 1–19Cite as

The Understanding of Meaningful Events in Gesture-Based Interaction

  • Chapter

Part of the book series: Studies in Computational Intelligence ((SCI,volume 332))

Abstract

Gesture-based interaction is becoming more and more available each day with the continuous advances and developments in acquisition technology and recognition algorithms as well as with the increasing availability of personal (mobile) devices, ambient media displays and interactive surfaces. Vision-based technology is the preferred choice when non-intrusiveness, unobtrusiveness and comfortable interactions are being sought. However, it also comes with the additional costs of difficult (unknown) scenarios to process and far-than-perfect recognition rates. The main challenge is represented by spotting and segmenting gestures in video media. Previous research has considered various events that specify when a gesture begins and when it ends in conjunction with location, time, motion, posture or various other segmentation cues. Therefore, video events identify, specify and segment gestures. Even more, when gestures are being correctly detected and recognized by the system with the appropriate feedback delivered to the human, the result is that gestures become themselves events in the human-computer dialogue: the commands were understood and the system reacted back.

This chapter addresses the double view of meaningful events: events that specify gestures together with intelligent algorithms that detect them in video sequences; gestures, that once recognized and accordingly interpreted by the system, become important events in the human-computer dialogue specifying the common understanding that was established. The chapter follows the duality aspect of events from the system as well as the human perspective contributing to the present understanding of gestures in human-computer interaction.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Austermann, A., Yamada, S., Funakoshi, K., Nakano, M.: How do users interact with a pet-robot and a humanoid. In: Proceedings of the 28th of the International Conference Extended Abstracts on Human Factors in Computing Systems, CHI EA 2010, Atlanta, Georgia, USA, April 10-15, pp. 3727–3732. ACM, New York (2010)

    Chapter  Google Scholar 

  2. Baudel, T., Beaudouin-Lafon, M.: Charade: remote control of objects using free-hand gestures. Communications of the ACM 36(7), 28–35 (1993)

    Article  Google Scholar 

  3. Baudisch, P., Chu, G.: Back-of-device interaction allows creating very small touch devices. In: Proceedings of the 27th International Conference on Human Factors in Computing Systems, CHI 2009, Boston, MA, USA, April 04-09, pp. 1923–1932. ACM, New York (2009)

    Chapter  Google Scholar 

  4. Caetano, T.S., Olabarriaga, S.D., Barone, D.A.C.: Do mixture models in chromaticity space improve skin detection? Pattern Recognition 36(12), 3019–3021 (2003)

    Article  MATH  Google Scholar 

  5. Cassell, J., Bickmore, T., Billinghurst, M., Campbell, L., Chang, K., Vilhjálmsson, H., Yan, H.: Embodiment in conversational interfaces: Rea. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: the CHI Is the Limit, CHI 1999, Pittsburgh, Pennsylvania, United States, May 15-20, pp. 520–527. ACM, New York (1999)

    Chapter  Google Scholar 

  6. Cassell, J.: Embodied conversational interface agents. ACM Commun. 43(4), 70–78 (2000)

    Article  Google Scholar 

  7. Cerlinca, T.I., Pentiuc, S.G., Vatavu, R.D., Cerlinca, M.C.: Hand posture recognition for human-robot interaction. In: Proceedings of the 2007 Workshop on Multimodal Interfaces in Semantic Interaction, WMISI 2007, Nagoya, Japan, November 15, pp. 47–50. ACM, New York (2007)

    Chapter  Google Scholar 

  8. Cho, K.-M., Jang, J.-H., Hong, K.-S.: Adaptive skin color filter. Pattern Recognition 34(5), 1067–1073 (2001)

    Article  MATH  Google Scholar 

  9. Demirdjian, D., Darrell, T.: 3-D Articulated Pose Tracking for Untethered Deictic Reference. In: Proceedings of International Conference on Multimodal Interfaces, ICMI 2002 (2002)

    Google Scholar 

  10. Dietz, P., Leigh, D.: DiamondTouch: a multi-user touch technology. In: Proceedings of the 14th annual ACM symposium on User interface software and technology (UIST 2001), Orlando, Florida, United States, pp. 219–226. ACM Press, New York (2001)

    Chapter  Google Scholar 

  11. Edlund, J., Gustafson, J., Heldner, M., Hjalmarsson, A.: Towards human-like spoken dialogue systems. Speech Commun. 50(8-9), 630–645 (2008)

    Article  Google Scholar 

  12. Erol, A., Bebis, G., Nicolescu, M., Boyle, R.D., Twombly, X.: Vision-based hand pose estimation: A review. Computer Vision and Image Understanding 108(1-2), 52–73 (2007)

    Article  Google Scholar 

  13. Freeman, W.T., Weissman, C.D.: Television Control by Hand Gestures. In: Proceedings of the 1st International Conference on Automatic Face and Gesture Recognition (1994)

    Google Scholar 

  14. Gratch, J., Wang, N., Okhmatovskaia, A., Lamothe, F., Morales, M., Van Der Werf, R.J., Morency, L.: Can virtual humans be more engaging than real ones? In: Jacko, J.A. (ed.) HCI 2007. LNCS, vol. 4552, pp. 286–297. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  15. Han, J.Y.: Low-cost multi-touch sensing through frustrated total internal reflection. In: Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology, UIST 2005, Seattle, WA, USA, October 23-26, pp. 115–118. ACM, New York (2005)

    Chapter  Google Scholar 

  16. Harling, P.A., Edwards, A.D.N.: Hand tension as a gesture segmentation cue. In: Progress in Gestural Interaction: Proceedings of Gesture Workshop 1996, pp. 75–87. Springer, Heidelberg (1997)

    Google Scholar 

  17. Heider, F., Simmel, M.: An Experimental Study of Apparent Behavior. The American Journal of Psychology 57(2), 243–259 (1944)

    Article  Google Scholar 

  18. Hilliges, O., Izadi, S., Wilson, A.D., Hodges, S., Garcia-Mendoza, A., Butz, A.: Interactions in the air: adding further depth to interactive tabletops. In: Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology, UIST 2009, Victoria, BC, Canada, October 04-07, pp. 139–148. ACM, New York (2009)

    Chapter  Google Scholar 

  19. Iannizzotto, G., Villari, M., Vita, L.: Hand tracking for human-computer interaction with Graylevel VisualGlove: turning back to the simple way. In: Proceedings of the 2001 Workshop on Perceptive User Interfaces, PUI 2001, Orlando, Florida, November 15-16, vol. 15, pp. 1–7. ACM, New York (2001)

    Chapter  Google Scholar 

  20. Ju, W., Takayama, L.: Approachability: How People Interpret Automatic Door Movement as Gesture. International Journal of Design 3(2) (2009)

    Google Scholar 

  21. Kanda, T., Hirano, T., Eaton, D., Ishiguro, H.: Interactive robots as social partners and peer tutors for children: a field trial. Hum.-Comput. Interact. 19(1), 61–84 (2004)

    Article  Google Scholar 

  22. Kanda, T., Kamasima, M., Imai, M., Ono, T., Sakamoto, D., Ishiguro, H., Anzai, Y.: A humanoid robot that pretends to listen to route guidance from a human. Auton. Robots 22(1), 87–100 (2007)

    Article  Google Scholar 

  23. Kolsch, M., Turk, M., Hollerer, T.: Vision-Based Interfaces for Mobility. In: Proceedings of the International Conference on Mobile and Ubiquitous Systems, MobiQuitous 2004 (2004)

    Google Scholar 

  24. Kolsch, M., Hollerer, T., DiVerdi, S.: HandVu: A New Machine Vision Library for Hand Tracking and Gesture Recognition, demo at ISWC/ISMAR (2004)

    Google Scholar 

  25. Kolsch, M., Turk, M.: Robust Hand Detection. In: Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition (2004)

    Google Scholar 

  26. Kolsch, M., Turk, M.: Hand tracking with Flocks of Features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2005)

    Google Scholar 

  27. Kry, P.G., Pihuit, A., Bernhardt, A., Cani, M.: HandNavigator: hands-on interaction for desktop virtual reality. In: Proceedings of the 2008 ACM Symposium on Virtual Reality Software and Technology, VRST 2008, Bordeaux, France, October 27-29, pp. 53–60. ACM, New York (2008)

    Chapter  Google Scholar 

  28. Lee, J.Y., Yoo, S.I.: An elliptical boundary model for skin color detection. In: Proceedings of the Int. Conf. on Imaging Science, Systems and Technology, Las Vegas, USA (2002)

    Google Scholar 

  29. Lee, J.C., Tan, D.S.: Using a low-cost electroencephalograph for task classification in HCI research. In: Proceedings of the 19th Annual ACM Symposium on User interface Software and Technology, UIST 2006, Montreux, Switzerland, October 15-18, pp. 81–90. ACM, New York (2006)

    Chapter  Google Scholar 

  30. Lee, J.C.: Hacking the Nintendo Wii Remote. IEEE Pervasive Computing 7(3), 39–45 (2008)

    Article  Google Scholar 

  31. Malik, S., Laszlo, J.: Visual touchpad: a two-handed gestural input device. In: Proceedings of the 6th International Conference on Multimodal Interfaces, State College, PA, USA, October 13-15, pp. 289–296. ACM, New York (2004)

    Chapter  Google Scholar 

  32. Marcel, S.: Hand posture recognition in a body-face centered space. In: CHI 1999 Extended Abstracts on Human Factors in Computing Systems, Pittsburgh, Pennsylvania, May 15-20, pp. 302–303. ACM, New York (1999)

    Chapter  Google Scholar 

  33. Moeslund, T.B., Granum, E.: A Survey of Computer Vision-Based Human Motion Capture. Computer Vision and Image Understanding 81(3), 231–268 (2001)

    Article  MATH  Google Scholar 

  34. Moeslund, T.B., Hilton, A., Kruger, V.: A survey of advances in vision-based human motion capture and analysis. Computer Vision and Image Understanding, Special Issue on Modeling People: Vision-based understanding of a person’s shape, appearance, movement and behaviour 104(2-3), 90–126 (2006)

    Google Scholar 

  35. Mutlu, B., Osman, S., Forlizzi, J., Hodgins, J., Kiesler, S.: Perceptions of ASIMO: an exploration on co-operation and competition with humans and humanoid robots. In: Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction, HRI 2006, Salt Lake City, Utah, USA, March 02-03, pp. 351–352. ACM, New York (2006)

    Chapter  Google Scholar 

  36. Nijholt, A., Tan, D., Allison, B., Milan, J.d.R., Graimann, B.: Brain-computer interfaces for HCI and games. In: CHI 2008 Extended Abstracts on Human Factors in Computing Systems, pp. 3925–3928. ACM, New York (2008)

    Google Scholar 

  37. Okuno, Y., Kanda, T., Imai, M., Ishiguro, H., Hagita, N.: Providing route directions: design of robot’s utterance, gesture, and timing. In: Proceedings of the 4th ACM/IEEE International Conference on Human Robot Interaction, HRI 2009, La Jolla, California, USA, March 09-13, pp. 53–60. ACM, New York (2009)

    Chapter  Google Scholar 

  38. Poppe, R.: Vision-based human motion analysis: An overview. Computer Vision and Image Understanding 108(1-2), 4–18 (2007)

    Article  Google Scholar 

  39. Reeves, B., Nass, C.: The Media Equation: how People Treat Computers, Television, and New Media Like Real People and Places. Cambridge University Press, Cambridge (1996)

    Google Scholar 

  40. Saffer, D.: Designing Gestural Interfaces. O’Reilly Media Inc., Sebastopol (2009)

    Google Scholar 

  41. Saponas, T.S., Tan, D.S., Morris, D., Balakrishnan, R., Turner, J., Landay, J.A.: Enabling always-available input with muscle-computer interfaces. In: Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology, UIST 2009, Victoria, BC, Canada, October 04-07, pp. 167–176. ACM, New York (2009)

    Chapter  Google Scholar 

  42. Saponas, T.S., Tan, D.S., Morris, D., Turner, J., Landay, J.A.: Making muscle-computer interfaces more practical. In: Proceedings of the 28th International Conference on Human Factors in Computing Systems, CHI 2010, Atlanta, Georgia, USA, April 10-15, pp. 851–854. ACM, New York (2010)

    Google Scholar 

  43. Sauvan, J., Lcuyer, A., Lotte, F., Casiez, G.: A performance model of selection techniques for P300-based brain-computer interfaces. In: Proceedings of CHI 2009, pp. 2205–2208. ACM, New York (2009)

    Google Scholar 

  44. Schlattman, M., Klein, R.: Simultaneous 4 gestures 6 DOF real-time two-hand tracking without any markers. In: Spencer, S.N. (ed.) Proceedings of the 2007 ACM Symposium on Virtual Reality Software and Technology, VRST 2007, Newport Beach, California, November 05-07, pp. 39–42. ACM, New York (2007)

    Chapter  Google Scholar 

  45. Jaimes, A., Sebe, N.: Multimodal human-computer interaction: A survey. Computer Vision and Image Understanding 108(1-2), 116–134 (2007)

    Article  Google Scholar 

  46. Sharma, N.K., Kurian, G.: Language, Thought and Communication. In: Krishnan, L., Patnaik, B.N., Sharma, N.K. (eds.) Aspects of human communication. Mittal Publications (1989)

    Google Scholar 

  47. Shi, J., Tomasi, C.: Good features to track. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (1994)

    Google Scholar 

  48. Sturman, D.J., Zeltzer, D.: A Survey of Glove-based Input. IEEE Computer Graphics and Applications 14(1), 30–39 (1994)

    Article  Google Scholar 

  49. Vatavu, R.D., Pentiuc, S.G., Chaillou, C., Grisoni, L., Degrande, S.: Visual Recognition of Hand Postures for Interacting with Virtual Environments. In: Proceedings of the 8th International Conference on Development and Application Systems - DAS 2006, Suceava, Romania, pp. 477–482 (2006)

    Google Scholar 

  50. Vatavu, R.D., Pentiuc, S.G.: Interactive Coffee Tables: Interfacing TV within an Intuitive, Fun and Shared Experience. In: Tscheligi, M., Obrist, M., Lugmayr, A. (eds.) EuroITV 2008. LNCS, vol. 5066, pp. 183–187. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  51. Vatavu, R.D., Grisoni, L., Pentiuc, S.G.: Gesture Recognition Based on Elastic Deformation Energies. In: Sales Dias, M., Gibet, S., Wanderley, M.M., Bastos, R. (eds.) GW 2007. LNCS (LNAI), vol. 5085, pp. 1–12. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  52. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 511–518 (2001)

    Google Scholar 

  53. Wagner, D., Billinghurst, M., Schmalstieg, D.: How real should virtual characters be? In: Proceedings of the 2006 ACM SIGCHI international Conference on Advances in Computer Entertainment Technology, ACE 2006, Hollywood, California, June 14-16, vol. 266, p. 57. ACM, New York (2006)

    Chapter  Google Scholar 

  54. Wang, R.Y., Popovic, J.: Real-time hand-tracking with a color glove. In: Hoppe, H. (ed.) ACM SIGGRAPH 2009 Papers, SIGGRAPH 2009, New Orleans, Louisiana, August 03-07, pp. 1–8. ACM, New York (2009)

    Chapter  Google Scholar 

  55. Wigdor, D., Leigh, D., Forlines, C., Shipman, S., Barnwell, J., Balakrishnan, R., Shen, C.: Under the table interaction. In: Proceedings of the 19th Annual ACM Symposium on User interface Software and Technology, UIST 2006, Montreux, Switzerland, October 15-18, pp. 259–268. ACM, New York (2006)

    Chapter  Google Scholar 

  56. Wigdor, D., Forlines, C., Baudisch, P., Barnwell, J., Shen, C.: Lucid touch: a see-through mobile device. In: Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology, UIST 2007, Newport, Rhode Island, USA, October 07-10, pp. 269–278. ACM, New York (2007)

    Chapter  Google Scholar 

  57. Wilson, A.D.: TouchLight: an imaging touch screen and display for gesture-based interaction. In: Proceedings of the 6th International Conference on Multimodal Interfaces, ICMI 2004, State College, PA, USA, October 13-15, pp. 69–76. ACM, New York (2004)

    Chapter  Google Scholar 

  58. Wilson, A.D.: PlayAnywhere: a compact interactive tabletop projection-vision system. In: Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology, UIST 2005, Seattle, WA, USA, October 23-26, pp. 83–92. ACM, New York (2005)

    Chapter  Google Scholar 

  59. Wilson, A., Robbins, D.C.: Playtogether: Playing games across multiple interactive tabletops. In: IUI Workshop on Tangible Play: Research and Design for Tangible and Tabletop Games (2006)

    Google Scholar 

  60. Wilson, A.D.: Robust computer vision-based detection of pinching for one and two-handed gesture input. In: Proceedings of the 19th Annual ACM Symposium on User Interface Software and Technology (UIST 2006), Montreux, Switzerland, pp. 255–258. ACM Press, New York (2006)

    Chapter  Google Scholar 

  61. Wu, M., Balakrishnan, R.: Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays. In: Proceedings of the 16th Annual ACM Symposium on User Interface Software and Technology, UIST 2003, Vancouver, Canada, November 02-05, pp. 193–202. ACM, New York (2003)

    Chapter  Google Scholar 

  62. Xiao, J.: Understanding the use and utility of anthropomorphic interface agents. In: CHI 2001 Extended Abstracts on Human Factors in Computing Systems, CHI 2001, Seattle, Washington, March 31-April 05, pp. 409–410. ACM, New York (2001)

    Chapter  Google Scholar 

  63. Yuksel, B.F., Donnerer, M., Tompkin, J., Steed, A.: A novel brain-computer interface using a multi-touch surface. In: Proceedings of the 28th International Conference on Human Factors in Computing Systems, CHI 2010, Atlanta, Georgia, USA, April 10-15, pp. 855–858. ACM, New York (2010)

    Google Scholar 

  64. Zhang, L.-G., Chen, Y., Fang, G., Chen, X., Gao, W.: A vision-based sign language recognition system using tied-mixture density HMM. In: ICMI 2004: Proceedings of the 6th International Conference on Multimodal Interfaces, State College, PA, USA, pp. 198–204. ACM Press, New York (2004)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University Stefan cel Mare of Suceava, 13 Universitatii, 720229, Suceava, Romania

    Radu-Daniel Vatavu

Authors
  1. Radu-Daniel Vatavu
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Computing, University of Dundee , DD1 4HN, Dundee, Scotland, UK

    Jianguo Zhang

  2. Department of Electronic & Electrical Engineering, The University of Sheffield, S1 3JD, Sheffield, UK

    Ling Shao

  3. Microsoft Research Asia , 49 Zhichun Road, 100190, Beijing, P.R. China

    Lei Zhang

  4. Digital Imaging Research Centre, Faculty of Computing, Information Systems and Mathematics, Kingston University, Penrhyn Road, Kingston upon Thames, KT1 2EE, Surrey, UK

    Graeme A. Jones

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Vatavu, RD. (2011). The Understanding of Meaningful Events in Gesture-Based Interaction. In: Zhang, J., Shao, L., Zhang, L., Jones, G.A. (eds) Intelligent Video Event Analysis and Understanding. Studies in Computational Intelligence, vol 332. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-17554-1_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-17554-1_1

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-17553-4

  • Online ISBN: 978-3-642-17554-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation