Abstract
Automatic visual recognition of gestures can be performed using either a trajectory-based or a history-based representation. The former characterises the gesture using 2D trajectories of the hands. The latter summarises image sequences using values computed from individual pixel histories. A direct experimental comparison of these two approaches is presented using skin colour as a common visual cue and recognition methods based on hidden Markov models, moment features and normalised template matching. Skin history images are proposed as a useful history-based representation. Results are reported on a database of sixty gestures and the relative advantages and disadvantages of the different methods are highlighted.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bobick, A., Davis, J.: The recognition of human movement using temporal templates. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(3), 257–267 (2001)
McKenna, S.J., Gong, S.: Gesture recognition for visually mediated interaction using probabilistic event trajectories. In: British Machine Vision Conference, Southampton, England, pp. 498–507 (September 1998)
Cutler, R., Turk, M.: View-based interpretation of real-time optical flow for gesture recognition. In: IEEE International Conference on Face & Gesture Recognition, Nara, Japan, pp. 416–421 (1998)
Ahmad, T., Taylor, C.J., Lanitis, A., Cootes, T.F.: Tracking and recognising hand gestures using statistical shape models. Images and Vision Computing 15(5), 345–352 (1997)
Yang, M.-H., Ahuja, N.: Extracting gestural motion trajectories. In: IEEE International Conference on Face & Gesture Recognition, pp. 10–15 (1998)
Pavlovic, V.I., Sharma, R., Huang, T.S.: Visual interpretation of hand gestures for human-computer interaction: A review. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(7), 677–695 (1997)
Gavrila, D.M.: The visual analysis of human movement: A survey. Computer Vision and Image Understanding 73(1), 82–98 (1999)
Wu, Y., Huang, T.S.: Vision-based gesture recognition: A review. Lecture Notes in Artificial Intelligence (1999)
Corradini, A., Gross, H.-M.: Implementation and comparison of three architectures for gesture recognition. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2000), Istanbul, Turkey, vol. IV, pp. 2361–2364 (June 2000)
Campbell, L.W., Becker, D.A., Azarbayejani, A., Bobick, A.F., Pentland, A.: Invariant features for 3-d gesture recognition. In: IEEE International Conference on Face & Gesture Recognition, Killington, USA, pp. 157–162 (1996)
Morrison, K., McKenna, S.J.: Automatic visual recognition of gestures made by motor-impaired computer users. Journal of Technology and Disability (2002); Special Issue on Rehabilitation Engineering (in press)
Swain, M.J., Ballard, D.H.: Color indexing. International Journal of Computer Vision 7(1), 11–32 (1991)
Kjeldsen, R., Kender, J.: Finding skin in color images. In: IEEE International Conference on Face & Gesture Recognition, Killington, Vermont, pp. 312–317 (October 1996)
Raja, Y., McKenna, S.J., Gong, S.: Tracking and segmenting people in varying lighting conditions using colour. In: IEEE International Conference on Face & Gesture Recognition, Nara, Japan, pp. 228–233 (1998)
Sigal, L., Sclaroff, S., Athitos, V.: Estimation and prediction of evolving color distributions for skin segmentation under varying illumination. In: IEEE Conference on Computer Vision and Pattern Recognition, South Carolina (June 2000)
Storring, M., Andersen, H.J., Granum, E.: Skin colour detection under changing lighting conditions. In: 7th Symposium on Intelligent Robotic Systems, Coimbra, Portugal (July 1999)
Zhu, X., Yang, J., Waibel, A.: Segmenting hands of arbitrary color. In: IEEE International Conference on Face & Gesture Recognition, Grenoble, France, March 2000, pp. 446–453 (2000)
Bobick, A.F., Davis, J.W.: An appearance-based representation of action. In: International Conference on Pattern Recognition, Vienna (1996)
Bobick, A.F.: Computers seeing action. In: British Machine Vision Conference, Edinburgh, Scotland, September 1996, pp. 13–22 (1996)
Bobick, A.F., Davis, J.W.: Real-time recognition of activity using temporal templates. In: Workshop on Applications of Computer Vision, Sarasoto, Florida (December 1996)
Davis, J.W.: Appearance-based motion recognition of human actions. Master’s thesis, Massachusetts Institute of Technology (July 1996)
Davis, J.W., Bobick, A.F.: The representation and recognition of action using temporal templates. In: IEEE Conference on Computer Vision and Pattern Recognition (1997)
Hu, M.: Visual pattern recognition by moment invariants. IRE Trans. Information Theory 8(2) (1962)
Li, K.: Reforming the theory of invariant moments for pattern recognition. Pattern Recognition 25(7), 723–730 (1992)
Ahmad, S.: A usable real-time 3D hand tracker. In: 28th Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, October 1994, pp. 1257–1261 (1994)
Imagawa, K., Lu, S., Igi, S.: Color-based hands tracking system for sign language recognition. In: IEEE International Conference on Face & Gesture Recognition, Nara, Japan, pp. 462–467 (1998)
Isard, M., Blake, A.: Icondensation: Unifying low-level and high-level tracking in a stochastic framework. In: European Conference on Computer Vision, pp. 893–908 (1998)
Sherrah, J., Gong, S.: Tracking discontinuous motion using Bayesian inference. In: European Conference on Computer Vision, Dublin, Ireland, pp. 150–166 (2000)
McKenna, S.J., Gong, S.: Tracking faces. In: IEEE International Conference on Face & Gesture Recognition, Killington, Vermont, USA (October 1996)
Min, B.-W., Yoon, H.-S., Soh, J., Ohashi, T., Ejima, T.: Visual recognition of static/dynamic gesture: Gesture-driven editing system. Journal of Visual Languages and Computing 10(3), 291–309 (1999)
Starner, T., Weaver, J., Pentland, A.: Real-time American Sign Language recognition using desk and wearable computer based video. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(12) (December 1998)
Wilson, A.D., Bobick, A.F.: Learning visual behaviour for gesture analysis. In: IEEE Symposium on Computer Vision, Coral Gables, Florida (November 1995)
Becker, D.A.: Sensei: A real-time recognition, feedback and training system for T’ai Chi gestures. Master’s thesis, Massachusetts Institute of Technology (May 1997)
Starner, T.E.: Visual recognition of American Sign Language using hidden Markov models. Master’s thesis, Massachusetts Institute of Technology (February 1995)
Iwai, Y., Shimizu, H., Yachida, M.: Real-time context-based gesture recognition using HMM and automaton. In: International Workshop on Recognition, Analysis and Tracking of Faces and Gestures in Real-Time Systems, Corfu, Greece (September 1999)
Nam, Y., Wohn, K.Y.: Recognition of space-time hand-gestures using hidden Markov model. In: ACM Symposium on Virtual Reality Software and Technology, pp. 51–58 (1996)
Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77(2), 257–286 (1989)
Young, S., Kershaw, D., Odell, J., Ollason, D., Vatlchev, V., Woodland, P.: The HTK Book. Cambridge University Engineering Department (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Morrison, K., McKenna, S.J. (2004). An Experimental Comparison of Trajectory-Based and History-Based Representation for Gesture Recognition. In: Camurri, A., Volpe, G. (eds) Gesture-Based Communication in Human-Computer Interaction. GW 2003. Lecture Notes in Computer Science(), vol 2915. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24598-8_15
Download citation
DOI: https://doi.org/10.1007/978-3-540-24598-8_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-21072-6
Online ISBN: 978-3-540-24598-8
eBook Packages: Springer Book Archive