Generalized Rank Conditions in Multiple View Geometry with Applications to Dynamical Scenes
In this paper, the geometry of a general class of projections from ℝn to ℝk (k < n) is examined, as a generalization of classic multiple view geometry in computer vision. It is shown that geometric constraints that govern multiple images of hyperplanes in ℝn, as well as any incidence conditions among these hyperplanes (such as inclusion, intersection, and restriction), can be systematically captured through certain rank conditions on the so-called multiple view matrix. All constraints known or unknown in computer vision for the projection from ℝ3 to ℝ2 are simply instances of this result. It certainly simplifies current efforts to extending classic multiple view geometry to dynamical scenes. It also reveals that since most new constraints in spaces of higher dimension are nonlinear, the rank conditions are a natural replacement for the traditional multilinear analysis. We also demonstrate that the rank conditions encode extremely rich information about dynamical scenes and they give rise to fundamental criteria for purposes such as stereopsis in n-dimensional space, segmentation of dynamical features, detection of spatial and temporal formations, and rejection of occluding T-junctions.
Keywordsmultiple view geometry rank condition multiple view matrix dynamical scenes segmentation formation detection occlusion structure from motion
Unable to display preview. Download preview PDF.
- 2.O. Faugeras, Q.-T. Luong, and T. Papadopoulo. Geometry of Multiple Images. The MIT Press, 2001.Google Scholar
- 3.R. Fossum, K. Huang, Y. Ma General rank conditions in multiple view geometry. UIUC, CSL Technical Report, UILU-ENG 01-2222 (DC-203), October 8, 2001.Google Scholar
- 4.R. Hartley. Lines and points in three views-a unified approach. In Proceedings of 1994 Image Understanding Workshop, pp. 1006–1016, Monterey, CA USA, 1994. OMNIPRESS.Google Scholar
- 5.R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge, 2000.Google Scholar
- 7.Y. Liu and T.S. Huang. Estimation of rigid body motion using straight line correspondences IEEE Workshop on Motion: Representation and Analysis, Kiawah Island, SC, May 1986.Google Scholar
- 9.Y Ma, K. Huang, and J. Košecká. New rank deficiency condition for multiple view geometry of line features. UIUC, CSL Technical Report, UILU-ENG 01-2209 (DC-201), May 8, 2001.Google Scholar
- 10.Y Ma, K. Huang, R. Vidal, J. Košecká, and S. Sastry. Rank conditions of multiple view matrix in multiple view geometry. UIUC, CSL Technical Report, UILU-ENG 01-2214 (DC-220), submitted to IJCV, June 18, 2001.Google Scholar
- 12.B. Triggs. Matching constraints and the joint image. In Proceedings of Fifth International Conference on Computer Vision, pp. 338–343, Cambridge, MA, USA, 1995. IEEE Comput. Soc. Press.Google Scholar
- 13.L. Wolf and A. Shashua. On projection matrices P k → P 2, k = 3,..., 6, and their applications in computer vision. In Proceedings of the Eighth International Conference on Computer Vision, pp. 412–419, Vancouver, Canada, 2001.Google Scholar
- 14.G. Sparr. A common framework for kinetic depth, reconstruction and motion for deformable objects. Proceedings of the Fourth European Conference on Computer Vision, pp. 471–482, Cambridge, England, 1994.Google Scholar