Constrained Structure and Motion from N Views of a Piecewise Planar Scene

Summary

This paper is about the problem of structure and motion recovery from N views of a rigid scene. Especially, we deal with the case of scenes containing planes, i.e. sets of coplanar points and focus on structure and motion estimation, which is one of the major steps of any reconstruction system. In more detail, we explore the strong constraint of multi-coplanarity, exploited only in a sub-optimal manner by most existing works. A typical example is to estimate an unconstrained structure and motion and then to fit planes and maybe correct 3D point positions to make them coplanar. In this paper, we present an approach to estimate camera motion and piecewise planar structure simultaneously and optimally: the result minimizes reprojection error, while satisfying the multi-coplanarity. The estimation problem is consistently parameterized using mixed 2D/3D entities. Experimental results show that the reconstruction is of clearly superior quality compared to traditional methods based only on points, even if the scene is not perfectly piecewise planar, and more accurate compared to other plane-based methods.