Abstract
We present a vision system for human-machine interaction that relies on a small wearable camera which can be mounted to common glasses. The camera views the area in front of the user, especially the hands. To evaluate hand movements for pointing gestures to objects and to recognise object reference, an approach relying on the integration of bottom-up generated feature maps and top-down propagated recognition results is introduced. In this vision system, modules for context free focus of attention work in parallel to a recognition system for hand gestures. In contrast to other approaches, the fusion of the two branches is not on the symbolic but on the sub-symbolic level by use of attention maps. This method is plausible from a cognitive point of view and facilitates the integration of entirely different modalities.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
G. Backer, B. Mertsching, and M. Bollmann. Data-and Model-Driven Gaze Control for an Active-Vision System. IEEE Trans. on Pattern Analysis and Machine Intelligence, 23(12):1415–1429, 2001.
C. Bauckhage, G. A. Fink, J. Fritsch, F. Kummert, F. Lömker, G. Sagerer, and S. Wachsmuth. An Integrated System for Cooperative Man-Machine Interaction. In IEEE Int.’l Symp. on Comp. Intelligence in Robotics and Automation, Banff, Canada, 2001.
V. Bruce and M. Morgan. Violations of Symmetry and Repetition in Visual Patterns. Psychological Review, 61:183–193, 1954.
D. Crevier and R. Lepage. Knowledge-based image understanding systems: A survey. Computer Vision and Image Understanding, 67(2):161–185, 1997.
M. Fislage, R. Rae, and H. Ritter. Using visual attention to recognize human pointing gestures in assembly tasks. In 7th IEEE Int’l Conf. Comp. Vision, 1999.
C. Harris and M. Stephens. A Combined Corner and Edge Detector. In Proc. 4th Alvey Vision Conf., pages 147–151, 1988.
G. Heidemann, D. Lücke, and H. Ritter. A System for Various Visual Classification Tasks Based on Neural Networks. In A. Sanfeliu et al., editor, Proc. 15th Int’l Conf. on Pattern Recognition ICPR 2000, Barcelona, volume I, pages 9–12, 2000.
G. Heidemann and H. Ritter. Efficient Vector Quantization Using the WTA-rule with Activity Equalization. Neural Processing Letters, 13(1):17–30, 2001.
G. Heidemann and H. Ritter. Visual Checking of Grasping Positions of a Three-Fingered Robot Hand. In G. Dorffner, H. Bischof, and K. Hornik, editors, Proc. ICANN 2001, pages 891–898. Springer-Verlag, 2001.
L. Itti, C. Koch, and E. Niebur. A Model of Saliency-Based Visual Attention for Rapid Scene Analysis. IEEE Trans. on Pattern Analysis and Machine Intelligence, 20(11):1254–1259, 1998.
I. Jolliffe. Principal Component Analysis. Springer Verlag, New York, 1986.
T. Kalinke and U. Handmann. Fusion of Texture and Contour Based Methods for Object Recognition. In IEEE Conf. on Intelligent Transportation Systems 1997, Stuttgart, 1997.
T. Kalinke and W. v. Seelen. Entropie als Maß des lokalen Informationsgehalts in Bildern zur Realisierung einer Aufmerksamkeitssteuerung. In B. Jähne et al., editor, Mustererkennung 1996. Springer, Heidelberg, 1996.
T. Kohonen. Self-organization and associative memory. In Springer Series in Information Sciences 8. Springer-Verlag Heidelberg, 1984.
P. J. Locher and C. F. Nodine. Symmetry Catches the Eye. In A. Levy-Schoen and J. K. O’Reagan, editors, Eye Movements: From Physiology to Cognition, pages 353–361. Elsevier Science Publishers B. V. (North Holland), 1987.
J. Moody and C. Darken. Learning with localized receptive fields. In Proc. of the 1988 Connectionist Models Summer School, pages 133–143. Morgan Kaufman Publishers, San Mateo, CA, 1988.
D. Reisfeld, H. Wolfson, and Y. Yeshurun. Context-Free Attentional Operators: The Generalized Symmetry Transform. Int’l J. of Computer Vision, 14:119–130, 1995.
H. J. Ritter, T. M. Martinetz, and K. J. Schulten. Neuronale Netze. Addison-Wesley, München, 1992.
T. D. Sanger. Optimal Unsupervised Learning in a Single-Layer Linear Feedforward Neural Network. Neural Networks, 2:459–473, 1989.
C. Schmid, R. Mohr, and C. Bauckhage. Evaluation of Interest Point Detectors. Int’l J. of Computer Vision, 37(2):151–172, 2000.
C. Theis, I. Iossifidis, and A. Steinhage. Image processing methods for interactive robot control. In Proc. IEEE Roman International Workshop on Robot-Human Interactive Communication, Bordeaux and Paris, France, 2001.
M. E. Tipping and C. M. Bishop. Mixtures of probabilistic principal component analyzers. Neural Computation, 11(2):443–482, 1999.
D. Walther, L. Itti, M. Riesenhuber, T. Poggio, and C. Koch. Attentional Selection for Object Recognition — a Gentle Way. In Proc. 2nd Workshop on Biologically Motivated Computer Vision (BMCV’02), Tübingen, Germany, 2002.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Heidemann, G., Rae, R., Bekel, H., Bax, I., Ritter, H. (2003). Integrating Context-Free and Context-Dependent Attentional Mechanisms for Gestural Object Reference. In: Crowley, J.L., Piater, J.H., Vincze, M., Paletta, L. (eds) Computer Vision Systems. ICVS 2003. Lecture Notes in Computer Science, vol 2626. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36592-3_3
Download citation
DOI: https://doi.org/10.1007/3-540-36592-3_3
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-00921-4
Online ISBN: 978-3-540-36592-1
eBook Packages: Springer Book Archive