Use your hand as a 3-D mouse, or, relative orientation from extended sequences of sparse point and line correspondences using the affine trifocal tensor

  • Lars Bretzner
  • Tony Lindeberg
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1406)


This paper addresses the problem of computing three-dimensional structure and motion from an unknown rigid configuration of point and lines viewed by an affine projection model. An algebraic structure, analogous to the trilinear tensor for three perspective cameras, is defined for configurations of three centered affine cameras. This centered affine trifocal tensor contains 12 non-zero coefficients and involves linear relations between point correspondences and trilinear relations between line correspondences. It is shown how the affine trifocal tensor relates to the perspective trilinear tensor, and how three-dimensional motion can be computed from this tensor in a straightforward manner. A factorization approach is also developed to handle point features and line features simultaneously in image sequences. This theory is applied to a specific problem in human-computer interaction of capturing three-dimensional rotations from gestures of a human hand. Besides the obvious application, this test problem illustrates the usefulness of the affine trifocal tensor in a situation where sufficient information is not available to compute the perspective trilinear tensor, while the geometry requires point correspondences as well as line correspondences over at least three views.


Point Feature User Equipment Line Feature Point Correspondence Centered Affine 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Beardsley, P., Torr, P. & Zisserman, A. (1996), 3D model acquistions from extended image sequences, in ‘4th ECCV', 683–695.Google Scholar
  2. Beardsley, P., Zisserman, A. & Murray, D. (1994), Navigation using affine structure from motion, in ‘3th ECCV', 85–96.Google Scholar
  3. Bigün, J., Granlund, G. H. & Wiklund, J. (1991), ‘Multidimensional orientation estimation with applications to texture analysis and optical flow', PAMI 13(8), 775–790.Google Scholar
  4. Bretzner, L. & Lindeberg, T. (1996), Feature tracking with automatic selection of spatial scales, ISRN KTH/NA/P-96/21-SE, KTH, Stockholm, Sweden.Google Scholar
  5. Bretzner, L. & Lindeberg, T. (1997), On the handling of spatial and temporal scales in feature tracking, in ‘Proc. 1st Scale-Space'97', Utrecht, Netherlands, 128–139.Google Scholar
  6. Bretzner, L. & Lindeberg, T. (1998), Use your hand as a 3-d mouse, or, relative orientation from extended sequences of sparse point and line correspondences using the affine trifocal tensor. Technical report to be published at KTH.Google Scholar
  7. Faugeras, O. (1992), What can be seen in three dimensions with a stereo rig?, in ‘2nd ECCV', 563–578.Google Scholar
  8. Faugeras, O. (1995), ‘Stratification of three-dimensional vision: Projective, affine and metric reconstructions', JOSA 12(3), 465–484.Google Scholar
  9. Faugeras, O. & Mourrain, B. (1995), On the geometry and algebra of the point and line correspondences between N images, in ‘5th ICCV', Cambridge, MA, 951–956.Google Scholar
  10. Förstner, W. A. & Gülch, E. (1987), A fast operator for detection and precise location of distinct points, corners and centers of circular features, in ‘ISPRS'.Google Scholar
  11. Hartley, R. (1995), A linear method for reconstruction from points and lines, in ‘5th ICCV', Cambridge, MA, 882–887.Google Scholar
  12. Heap, T. & Hogg, D. (1996), Towards 3D hand tracking using a deformable model, in ‘Int. Conf. Autom. Face and Gesture Recogn., Killington, Vermont, 140–145.Google Scholar
  13. Heyden, A. (1995), Reconstruction from image sequences by means of relative depth, in ‘5th ICCV', Cambridge, MA, 57–66.Google Scholar
  14. Heyden, A., Sparr, G. & åström, K. (1997), Perception and action using multilinear forms, in ‘Proc. AFPAC'97', Kiel, Germany, 54–65.Google Scholar
  15. Huang, T. S. & Lee, C. H. (1989), ‘Motion and structure from orthographic projection', IEEE-PAMI 11(5), 536–540.Google Scholar
  16. Huang, T. S. & Netravali, A. N. (1994), ‘Motion and structure from feature correspondences: A review', Proc. IEEE 82, 251–268.CrossRefGoogle Scholar
  17. Koenderink, J. J. (1984), ‘The structure of images', Biol. Cyb. 50, 363–370.zbMATHMathSciNetCrossRefGoogle Scholar
  18. Koenderink, J. J. & van Doorn, A. J. (1991), ‘Affine structure from motion', JOSA 377–385.Google Scholar
  19. Lee, J. & Kunii, T. L. (1995), ‘Model-based analysis of hand posture', Computer Graphics and Applications pp. 77–86.Google Scholar
  20. Lindeberg, T. (1994), Scale-Space Theory in Computer Vision, Kluwer, Netherlands.Google Scholar
  21. Lindeberg, T. (1996), Edge detection and ridge detection with automatic scale selection, in ‘CVPR'96', 465–470.Google Scholar
  22. Lindeberg, T. & Bretzner, L. (1998), Visuellt människa-maskin-gränssnitt för tredimensionell orientering. Patent application.Google Scholar
  23. Longuet-Higgins, H. C. (1981), ‘A computer algorithm for reconstructing a scene from two projections', Nature 293, 133–135.CrossRefGoogle Scholar
  24. Maybank, S. (1992), Theory of Reconstruction from Image Motion, Springer-Verlag.Google Scholar
  25. McLauchlan, P., Reid, I. & Murray, D. (1994), Recursive affine structure and motion from image sequences, in ‘3th ECCV', Vol. 800, 217–224.Google Scholar
  26. Morita, T. & Kanade, T. (1997), ‘A sequential factorization method for recovering shape and motion from image streams', IEEE-PAMI 19(8), 858–867.Google Scholar
  27. Mundy, J. L. & Zisserman, A., eds (1992), Geometric Invariance in Computer Vision, MIT Press.Google Scholar
  28. Quan, L. & Kanade, T. (1997), ‘Affine structure from line correspondences with uncalibrated affine cameras', IEEE-PAMI 19(8), 834–845.Google Scholar
  29. Shapiro, L. S. (1995), Affine analysis of image sequences, Cambridge University Press.Google Scholar
  30. Shashua, A. (1995), ‘Algebraic functions for recognition', IEEE-PAMI 17(8), 779–789.Google Scholar
  31. Shashua, A. (1997), Trilinear tensor: The fundamental construct of multiple-view geometry and its applications, in ‘Proc. AFPAC'97', Kiel, Germany, 190–206.Google Scholar
  32. Spetsakis, M. E. & Aloimonos, J. (1990), ‘Structure from motion using line correspondences', IJCV 4(3), 171–183.CrossRefGoogle Scholar
  33. Sturm, P. & Triggs, B. (1996), A factorization based algorithm for multi-image projective structure and motion, in ‘4th ECCV', Vol. 1064, 709–720.Google Scholar
  34. Tomasi, C. & Kanade, T. (1992), ‘Shape and motion from image streams under orthography: A factorization method,’ IJCV 9(2), 137–154.CrossRefGoogle Scholar
  35. Torr, P. H. S. (1995), Motion Segmentation and Outlier Detection, PhD thesis, Univ. of Oxford.Google Scholar
  36. Ullman, S. (1979), The Interpretation of Visual Motion, MIT Press.Google Scholar
  37. Ullman, S. & Basri, R. (1991), ‘Recognition by linear combinations of models', IEEEPAMI 13(10), 992–1006.Google Scholar
  38. Weng, J., Huang, T. S. & Ahuja, N. (1992), ‘Motion and structure from line correspondences: Closed form solution and uniqueness results', IEEE-PAMI 14(3), 318–336.Google Scholar
  39. Xu, G. & Zhang, Z., eds (1997), Epipolar Geometry in Stereo, Motion and Object Recognition: A Unified Approach, Kluwer, Netherlands.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1998

Authors and Affiliations

  • Lars Bretzner
    • 1
  • Tony Lindeberg
    • 1
  1. 1.Dept. of Numerical Analysis and Computing ScienceComputational Vision and Active Perception Laboratory (CVAP)StockholmSweden

Personalised recommendations