Deictic Gestures with a Time-of-Flight Camera

  • Martin Haker
  • Martin Böhme
  • Thomas Martinetz
  • Erhardt Barth
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5934)


We present a robust detector for deictic gestures based on a time-of-flight (TOF) camera, a combined range and intensity image sensor. Pointing direction is used to determine whether the gesture is intended for the system at all and to assign different meanings to the same gesture depending on pointing direction. We use the gestures to control a slideshow presentation: Making a “thumbs-up” gesture while pointing to the left or right of the screen switches to the previous or next slide. Pointing at the screen causes a “virtual laser pointer” to appear. Since the pointing direction is estimated in 3D, the user can move freely within the field of view of the camera after the system was calibrated. The pointing direction is measured with an absolute accuracy of 0.6 degrees and a measurement noise of 0.9 degrees near the center of the screen.


Visual Feedback Initial Guess Segmented Image Foreground Pixel Temporal Smoothing 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Oggier, T., Büttgen, B., Lustenberger, F., Becker, G., Rüegg, B., Hodac, A.: SwissRangerTM SR3000 and first experiences based on miniaturized 3D-TOF cameras. In: Ingensand, K. (ed.) Proc. 1st Range Imaging Research Day, Zurich, pp. 97–108 (2005)Google Scholar
  2. 2.
    Baudel, T., Beaudouin-Lafon, M.: CHARADE: Remote control of objects using free-hand gestures. Communications of the ACM 36, 28–35 (1993)CrossRefGoogle Scholar
  3. 3.
    Hofemann, N., Fritsch, J., Sagerer, G.: Recognition of deictic gestures with context. In: Rasmussen, C.E., Bülthoff, H.H., Schölkopf, B., Giese, M.A. (eds.) DAGM 2004. LNCS, vol. 3175, pp. 334–341. Springer, Heidelberg (2004)Google Scholar
  4. 4.
    Moeslund, T.B., Nøregaard, L.: Recognition of deictic gestures for wearable computing. In: Gibet, S., Courty, N., Kamp, J.-F. (eds.) GW 2005. LNCS (LNAI), vol. 3881, pp. 112–123. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  5. 5.
    Moeslund, T.B., Granum, E.: Modelling and estimating the pose of a human arm. Machine Vision and Applications 14, 237–247 (2003)CrossRefGoogle Scholar
  6. 6.
    Haker, M., Böhme, M., Martinetz, T., Barth, E.: Geometric invariants for facial feature tracking with 3D TOF cameras. In: Proceedings of the IEEE International Symposium on Signals, Circuits & Systems (ISSCS), Iasi, Romania, vol. 1, pp. 109–112 (2007)Google Scholar
  7. 7.
    Haker, M., Martinetz, T., Barth, E.: Multimodal sparse features for object detection. In: Alippi, C., Polycarpou, M., Panayiotou, C., Ellinas, G. (eds.) ICANN 2009. LNCS, vol. 5769, pp. 923–932. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  8. 8.
    Kalman, R.E.: A new approach to linear filtering and prediction problems. Transactions of the ASME, Series D, Journal of Basic Engineering 82, 35–45 (1960)Google Scholar
  9. 9.
    Nickel, K., Stiefelhagen, R.: Pointing gesture recognition based on 3D-tracking of face, hands and head orientation. In: International Conference on Multimodal Interfaces, pp. 140–146 (2003)Google Scholar
  10. 10.
    Gudmundsson, S.A., Aanæs, H., Larsen, R.: Effects on measurement uncertainties of time-of-flight cameras. In: Proceedings of the IEEE International Symposium on Signals, Circuits & Systems (ISSCS), vol. 1, pp. 1–4 (2007)Google Scholar
  11. 11.
    Kranstedt, A., Lücking, A., Pfeiffer, T., Rieser, H., Wachsmuth, I.: Deixis: How to Determine Demonstrated Objects Using a Pointing Cone. In: Gibet, S., Courty, N., Kamp, J.-F. (eds.) GW 2005. LNCS (LNAI), vol. 3881, pp. 300–311. Springer, Heidelberg (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Martin Haker
    • 1
  • Martin Böhme
    • 1
  • Thomas Martinetz
    • 1
  • Erhardt Barth
    • 1
  1. 1.Institute for Neuro- and BioinformaticsUniversity of LübeckLübeckGermany

Personalised recommendations