Piecewise-Planar StereoScan:Structure and Motion from Plane Primitives

  • Carolina Raposo
  • Michel Antunes
  • Joao P. Barreto
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8690)


This article describes a pipeline that receives as input a sequence of images acquired by a calibrated stereo rig and outputs the camera motion and a Piecewise-Planar Reconstruction (PPR) of the scene. It firstly detects the 3D planes viewed by each stereo pair from semi-dense depth estimation. This is followed by estimating the pose between consecutive views using a new closed-form minimal algorithm that relies in point correspondences only when plane correspondences are insufficient to fully constrain the motion. Finally, the camera motion and the PPR are jointly refined, alternating between discrete optimization for generating plane hypotheses and continuous bundle adjustment. The approach differs from previous works in PPR by determining the poses from plane-primitives, by jointly estimating motion and piecewise-planar structure, and by operating sequentially, being suitable for applications of SLAM and visual odometry. Experiments are carried in challenging wide-baseline datasets where conventional point-based SfM usually fails.


Structure and Motion Piecewise-Planar Reconstruction 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    P.F. Alcantarilla, C. Beall, F. Dellaert.: Large-scale dense 3D reconstruction from stereo imagery. In: 5th Workshop on Planning, Perception and Navigation for Intelligent Vehicles (PPNIV13) (2013)Google Scholar
  2. 2.
    Antunes, M., Barreto, J.P.: Semi-dense piecewise planar stereo reconstruction using symstereo and pearl. In: 3DimPVT (2012)Google Scholar
  3. 3.
    Antunes, M., Barreto, J.P., Zabulis, X.: Plane surface detection and reconstruction using induced stereo symmetry. In: BMVC (2011)Google Scholar
  4. 4.
    Antunes, M.: Phd thesis: Stereo reconstruction using induced symmetry and 3D scene priors (2014),
  5. 5.
    Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (surf). Comput. Vis. Image Underst. 110(3), 346–359 (2008)CrossRefGoogle Scholar
  6. 6.
    Delong, A., Osokin, A., Isack, H., Boykov, Y.: Fast approximate energy minimization with label costs. International Journal of Computer Vision 96(1), 1–27 (2012), CrossRefzbMATHMathSciNetGoogle Scholar
  7. 7.
    Dunn, E., Clipp, B., Frahm, J.M.: A geometric solver for calibrated stereo egomotion. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 1187–1194 (November 2011)Google Scholar
  8. 8.
    Fraundorfer, F., Tanskanen, P., Pollefeys, M.: A minimal case solution to the calibrated relative pose problem for the case of two known orientation angles. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 269–282. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  9. 9.
    Furukawa, Y., Curless, B., Seitz, S., Szeliski, R.: Manhattan-world stereo. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 1422–1429 (June 2009)Google Scholar
  10. 10.
    Furukawa, Y., Curless, B., Seitz, S., Szeliski, R.: Reconstructing building interiors from images. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 80–87 (September 2009)Google Scholar
  11. 11.
    Gallup, D., Frahm, J.M., Pollefeys, M.: Piecewise planar and non-planar stereo for urban scene reconstruction. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1418–1425 (June 2010)Google Scholar
  12. 12.
    Geiger, A., Ziegler, J., Stiller, C.: Stereoscan: Dense 3D reconstruction in real-time. In: Intelligent Vehicles Symposium (IV) (2011)Google Scholar
  13. 13.
    Grimson, W., Lozano-Pérez, T.: Model-based recognition and localization from sparse range or tactile data. International Journal of Robotics Research 3(3), 3–35 (1984), CrossRefGoogle Scholar
  14. 14.
    Horn, B.K.P.: Closed-form solution of absolute orientation using unit quaternions. J. Opt. Soc. Am. A 4(4), 629–642 (1987)CrossRefMathSciNetGoogle Scholar
  15. 15.
    Isack, H., Boykov, Y.: Energy-based geometric multi-model fitting. IJCV (2012)Google Scholar
  16. 16.
    Kazik, T., Kneip, L., Nikolic, J., Pollefeys, M., Siegwart, R.: Real-time 6D stereo visual odometry with non-overlapping fields of view. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1529–1536 (June 2012)Google Scholar
  17. 17.
    Lazic, N., Frey, B.J., Aarabi, P.: Solving the uncapacitated facility location problem using message passing algorithms. Journal of Machine Learning Research (2010)Google Scholar
  18. 18.
    Micusik, B., Kosecka, J.: Piecewise planar city 3d modeling from street view panoramic sequences. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 2906–2912 (June 2009)Google Scholar
  19. 19.
    Nister, D., Naroditsky, O., Bergen, J.: Visual odometry. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 1, pp. I-652–I-659 (June 2004)Google Scholar
  20. 20.
    Pathak, K., Birk, A., Vaškevičius, N., Poppinga., J.: Fast registration based on noisy planes with unknown correspondences for 3D mapping. IEEE Transactions on Robotics 26(3), 424–441 (2010)CrossRefGoogle Scholar
  21. 21.
    Raposo, C., Lourenco, M., Antunes, M., Barreto, J.P.: Plane-based odometry using an rgb-d camera, pp. 1–11 (September 2013)Google Scholar
  22. 22.
    Sinha, S., Steedly, D., Szeliski, R.: Piecewise planar stereo for image-based rendering. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 1881–1888 (September)Google Scholar
  23. 23.
    Snavely, N., Seitz, S.M., Szeliski, R.: Modeling the world from internet photo collections. Int. J. Comput. Vision 80(2), 189–210 (2008), CrossRefGoogle Scholar
  24. 24.
    Taguchi, Y., Jian, Y.D., Ramalingam, S., Feng, C.: Slam using both points and planes for hand-held 3d sensors. In: Proceedings of the 2012 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2012, pp. 321–322. IEEE Computer Society, Washington, DC (2012), Google Scholar
  25. 25.
    Vasconcelos, F., Barreto, J., Nunes, U.: A minimal solution for the extrinsic calibration of a camera and a laser-rangefinder. IEEE Transactions on Pattern Analysis and Machine Intelligence PP(99), 1 (2012)Google Scholar
  26. 26.
    Werner, T., Zisserman, A.: New techniques for automated architectural reconstruction from photographs. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part II. LNCS, vol. 2351, pp. 541–555. Springer, Heidelberg (2002), CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Carolina Raposo
    • 1
  • Michel Antunes
    • 1
  • Joao P. Barreto
    • 1
  1. 1.Institute of Systems and RoboticsUniversity of CoimbraCoimbraPortugal

Personalised recommendations