Hand-Gesture Recognition as a 3-D Input Technique

  • Ulrich Bröckl-Fox
Conference paper
Part of the Eurographics book series (EUROGRAPH)


3-D input techniques based on vision systems are presented: human extremities (hands and head) are observed by a video camera. Certain 2-D parameters of the depicted extremities are used as input parameters for 3-D metaphors.

The 2-D parameters needed and the real-time, contour-based computation of these on standard hardware is presented. Among these parameters, stable hand-shape classification is the key to simple and robust 3-D metaphors. A filter algorithm to separate hand and forearm is described. Different mappings of these parameters into 3-D movements and their metaphorical meanings behind these mappings are shown.

To evaluate these approaches a network-distributed Virtual Squash game was implemented, and the effectiveness of the video-based metaphors was compared directly to that of the space-ball. The results reveal that the precision of translations is slightly worse, the number of hits and the velocity of the Virtual squash-racket is slightly better, but the precision of rotations is supreme for video-based hand-gesture input.


Virtual Reality Euler Angle Input Device Abduction Angle Desktop Application 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    A. Azarbayejani, T. Starner, B. Horowitz, A. Pentland. Visually Controlled Graph-ics. Transactions on Pattern Analysis and Machine Intelligence, 15 (6): 602–605, June 1993.Google Scholar
  2. [2]
    R. Azuma, M. Ward, R. Bennett, S. Gottschalk, H. Fuchs. A Demonstrated Optical Tracker With Scalable Work Area for Head-Mounted Display Systems. In Proceedings of the 1992 Symposium on Interactive 3D Graphics, S. 43–51, 1992.Google Scholar
  3. [3]
    R. Bolt, E. Herranz. Two-Handed Gesture in Multi-Modal Natural Dialog. In Symposium on User Interface Software and Technology, S. 7–14. ACM, November 1992.Google Scholar
  4. [4]
    U. Bröckl, A. J. Klingert, A. Schmitt. Towards Standardized User Interfaces for Three-Dimensional Interaction: the Development of 3D Widgets Based on PEX and Motif. In Proceedings of the Fourth Annual Conference of the European X User Group (EXUG), 1992.Google Scholar
  5. [5]
    U. Bröckl-Fox. Untersuchung neuer, gestenbasierter Methoden für die SD-Interaktion. Verlag Shaker, Aachen, FRG, 1995.Google Scholar
  6. [6]
    U. Bröckl-Fox, L. Kettner, A. Klingert, L. Kobbelt. Artificial Life and Virtual Reality, Kapitel XII, Using Three Dimensional Hand-Gesture Recognition as a New 3D Input Technique, S. 173–187. Wiley and Sons, 1994.Google Scholar
  7. [7]
    Michael Chen, S. Joy Mountford, Abigail Sellen. A Study in Interactive 3-D Rotation Using 2-D Control Devices. Computer Graphics, 22 (4): 121–129, August 1988.CrossRefGoogle Scholar
  8. [8]
    M. Dai, P. Baylou, M. Najim. An Efficient Algorithm for Computation of Shape Moments from Run-Length Codes or Chain Codes. Pattern Recognition, 25 (10): 1119–1128, 1992.CrossRefGoogle Scholar
  9. [9]
    T. Darrell, A. Pentland. Space-Time Gestures. In IEEE Conference Computer Vision and Pattern Recognition, S. 294–299, New York City, June 15–17 1993.Google Scholar
  10. [10]
    Wolfgang Feiger. How Interactive Visualization Can Benefit from Multidimensional Input Devices. In Proceedings of the SPIE-The International Society for Optical Engineering, Band 1668, Visual Data Interpretation, S. 15–24, 1992.Google Scholar
  11. [11]
    Martin Friedmann, Thad Starner, Alex Pentland. Device Synchronisation Using an Optimal Linear Filter. In Proceedings of the 1992 Symposium on Interactive 3D Graphics, Special Issue of Computer Graphics, Band 26, S. 57–62, 1992.CrossRefGoogle Scholar
  12. [12]
    R. C. Gonzalez, P. Wintz. Digital Image Processing. Addison-Wesley, Reading, MA, 2 edition, 1987.Google Scholar
  13. [13]
    A. Goshtasby, F. Cheng, B. A. Barsky. B-Spline Curves and Surfaces Viewed as Digital Filters. Computer Vision, Graphics, and Image Processing, 52: 264–275, 1990.CrossRefGoogle Scholar
  14. [14]
    Andrew J. Hanson. The Rolling Ball. In David Kirk (Hrsg.), Graphics Gems III, S. 51–60. Academic Press, 1992.Google Scholar
  15. [15]
    H. Heuser. Lehrbuch der Analysis, Teil 2. B. G. Teubner Stuttgart, 1983.Google Scholar
  16. [16]
    X. Y. Jiang, H. Bunke. Ein konturbasierter Ansatz zur Berechnung von Momenten. In Informatik Fachberichte 290, S. 143–150. Bernd Radig, October 1991.Google Scholar
  17. [17]
    B. Klaus, P. Horn. Robot Vision. The MIT Press, McGraw-Hill Book Company, 1986.Google Scholar
  18. [18]
    M. W. Krueger. Artificial Reality. Addison-Wesley, 1981.Google Scholar
  19. [19]
    Jia-Guu Leu. Computing A Shape’s Moments from Its Boundary. Pattern Recognition, 24 (10): 949–957, 1991.CrossRefMathSciNetGoogle Scholar
  20. [20]
    P. Maes, T. Darrell, B. Blumberg, S. Pentland. Interacting with Animated Au-tonomous Agents. In Announced paper for SIGGRAPH 94, Orlando, FL, 1994.Google Scholar
  21. [21]
    C. Maggioni. A Novel Gestural Input Device for Virtual Reality. In Proceedings of IEEE 1993 Virtual Reality Annual International Symposium, VRAIS, S. 118–124, 1993.CrossRefGoogle Scholar
  22. [22]
    C. Maggioni. Non Immersive Control of Virtual Environments. In Virtual Reality, Anwendungen & Trends, IPA/IAO-Forum, S. 129–142, Stuttgart-Vaihingen, 9–10 Februar 1994. Springer Verlag.Google Scholar
  23. [23]
    J. S. C. Yuan. Mukundan. Estimation of Quaternion Parameters from Two Dimensional Image Moments. Graphical Models and Image Processing, 54: 345–350, 1992.CrossRefGoogle Scholar
  24. [24]
    B. A. Myers, M. Rosson. Survey on User Interface Programming. Number (#77629) in RC 17624. IBM Research Division, January 1992.Google Scholar
  25. [25]
    M. Nadler. A Note on Coefficients of Compass Mask Coefficients. Computer Vision, Graphic, and Image Processing, 51: 96–101, 1990.CrossRefGoogle Scholar
  26. [26]
    P. Nahim. The Theory of Measurement of a Silhouette Description for Image Processing and Recognition. Pattern Recognition, 6 (2): 85–95, 1974.CrossRefGoogle Scholar
  27. [27]
    E Quek. Toward a Vision-Based Hand Gesture Interface. In Conference on Virtual Reality Software and Technology, Singapore, August 23–26 1994.Google Scholar
  28. [28]
    J. Rehg, T. Kanade. DigitEyes: Vision-Based Human Hand Tracking. File: CMU-CS-93–220. ps. Zftp:reports. adm. cs. emu. edu:/usr0/anon/l993, S. 1–17, 1993.Google Scholar
  29. [29]
    I. Rothe, K. Voss. Orientierungsbestimmung von Objekten durch Momentinvarianten, S. 42–49. Informatik Aktuell. Springer Verlag: S. Fuchs, R. Hoffmann, 1992.Google Scholar
  30. [30]
    D. Rubine. Specifying Gestures by Example. Computer Graphics, 25 (4): 329–337, 1991.CrossRefGoogle Scholar
  31. [31]
    C. Ware, S. Osborne. Exploration and Virtual Camera Control in Virtual Three Dimensional Environments. In ACM Transactions On Computer Graphics, S. 175–183, Oct. 1990.Google Scholar

Copyright information

© Springer-Verlag/Wien 1995

Authors and Affiliations

  • Ulrich Bröckl-Fox
    • 1
  1. 1.Institut für Betriebs- und Dialogsysteme Lehrstuhl Prof. Dr. A. SchmittUniversität KarlsruheKarlsruheGermany

Personalised recommendations