3D Modelling of Static Environments Using Multiple Spherical Stereo

  • Hansung Kim
  • Adrian Hilton
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6554)


We propose a 3D modelling method from multiple pairs of spherical stereo images. A static environment is captured as a vertical stereo pair with a rotating line scan camera at multiple locations and depth fields are extracted for each pair using spherical stereo geometry. We propose a new PDE-based stereo matching method which handles occlusion and over-segmentation problem in highly textured regions. In order to avoid cumbersome camera calibration steps, we extract a 3D rigid transform using feature matching between views and fuse all models into one complete mesh. A reliable surface selection algorithm for overlapped surfaces is proposed for merging multiple meshes in order to keep surface details while removing outliers. The performances of the proposed algorithms are evaluated against ground-truth from LIDAR scans.


Environment modelling Spherical stereo PDE-based disparity estimation Multiple stereo reconstruction 


  1. 1.
    Lemmens, M.: Airborne LiDAR Sensor. GIM International 21 (2007)Google Scholar
  2. 2.
    Strecha, C., Hansen, W., Gool, L.V., Fua, P., Thoennessen, U.: On Benchmarking Camera Calibration and Multi-View Stereo for High Resolution Imagery. In: Proc. CVPR, pp. 1–8 (2008)Google Scholar
  3. 3.
    Furukawa, Y., Ponce, J.: Accurate, Dense, and Robust Multi-View Stereopsis. In: Proc. CVPR (2007)Google Scholar
  4. 4.
    Vu, H., Keriven, R., Labatut, P., Pons, J.P.: Towards high-resolution large-scale multi-view stereo. In: Proc. CVPR, pp. 1430–1437 (2009)Google Scholar
  5. 5.
    Salman, N., Yvinec, M.: Surface Reconstruction from Multi-View Stereo. In: Proc. ACCV (2009)Google Scholar
  6. 6.
    Agarwal, S., Snavely, N., Simon, I., Seitz, S.M., Szeliski, R.: Building Rome in a Day. In: Proc. ICCV (2009)Google Scholar
  7. 7.
    Pollefeys, M., Koch, R., Vergauwen, M., Gool, L.V.: Automated reconstruction of 3D scenes from sequences of images. ISPRS 55, 251–267 (2000)CrossRefGoogle Scholar
  8. 8.
    Cornelis, N., Leibe, B., Cornelis, K., Gool, L.V.: 3D urban scene modeling integrating recognition and reconstruction. IJCV 78, 121–141 (2008)CrossRefGoogle Scholar
  9. 9.
    Pollefeys, M., et al.: Detailed real-time urban 3D reconstruction from video. IJCV 78, 143–167 (2008)CrossRefGoogle Scholar
  10. 10.
    Nayar, S.: Catadioptric Omnidirectional Camera. In: Proc. CVPR, pp. 482–488 (1997)Google Scholar
  11. 11.
    Li, S.: Full-View Spherical Image Camera. In: Proc. ICPR, pp. 386–390 (2006)Google Scholar
  12. 12.
    Feldman, D., Weinshall, D.: Realtime IBR with Omnidirectional Crossed-Slits Projection. In: Proc. ICCV, pp. 839–845 (2005)Google Scholar
  13. 13.
    Kim, H., Hilton, A.: 3D Environment Modelling Using Spherical Stereo Imaging. In: Proc. 3DIM Workshop in ICCV (2009)Google Scholar
  14. 14.
    Boström, G., Fiocco, M., Puig, D., Rossini, A., Gonçalves, J.G.M., Sequeira, V.: Acquisition, Modelling and Rendering of Very Large Urban Environments. In: Proc. 3DPVT, pp. 191–198 (2004)Google Scholar
  15. 15.
    Asai, T., Kanbara, M., Yokoya, N.: 3D Modeling of Outdoor Environments by Integrating Omnidirectional Range and Color Images. In: Proc. 3DIM, pp. 447–454 (2005)Google Scholar
  16. 16.
    Bay, H., Ess, A., Tuytelaars, T., Gool, L.V.: SURF: Speeded Up Robust Features. CVIU 110, 346–359 (2008)Google Scholar
  17. 17.
    Li, S., Fukumori, K.: Spherical Stereo for the Construction of Immersive VR Environment. In: Proc. IEEE VR, pp. 217–222 (2005)Google Scholar
  18. 18.
    Scharstein, D., Szeliski, R.: A Taxonomy and Evaluation of Dense Two-frame Stereo Correspondence Algorithms. IJCV 47, 7–42 (2002)CrossRefzbMATHGoogle Scholar
  19. 19.
    Alvarez, L., Deriche, R., Sánchez, J., Weickert, J.: Dense Disparity Map Estimation Respecting Image Discontinuities: A PDE and Scale-Space Based Approach. Journal of Visual Comm. and Image Repr. 13, 3–21 (2002)CrossRefGoogle Scholar
  20. 20.
    Weickert, J.: A Review of Nonlinear Diffusion Filtering. In: ter Haar Romeny, B.M., Florack, L.M.J., Viergever, M.A. (eds.) Scale-Space 1997. LNCS, vol. 1252, pp. 3–28. Springer, Heidelberg (1997)Google Scholar
  21. 21.
    Zimmer, H., Bruhn, A., Valgaerts, L., Breuß, M., Weickert, J., Rosenhahn, B., Seidel, H.P.: PDE-based anisotropic disparity-driven stereo vision. In: Proc. VMV, pp. 263–272 (2008)Google Scholar
  22. 22.
    Sun, D., Roth, S., Lewis, J., Black, M.: Learning Optical Flow. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 83–97. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  23. 23.
    Ben-Ari, R., Sochen, N.: Variational Stereo Vision with Sharp Discontinuities and Occlusion Handling. In: Proc. ICCV, pp. 1–7 (2007)Google Scholar
  24. 24.
    Alvarez, L., Deriche, R., Papadopoulo, T., Sánchez, J.: Symmetrical Dense Optical Flow Estimation with Occlusions Detection. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part I. LNCS, vol. 2350, pp. 721–735. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  25. 25.
    Ince, S., Konrad, J.: Occlusion-aware optical flow estimation. IEEE Trans. Image Proc. 17, 1443–1451 (2008)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High Accuracy Optical Flow Estimation Based on a Theory for Warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  27. 27.
    Kim, H., Sohn, K.: Hierarchical Depth Estimation for Image Synthesis in Mixed Reality. In: Proc. SPIE Electronic Imaging, pp. 544–553 (2003)Google Scholar
  28. 28.
    Middlebury stereo and multi-view benchmarking site,
  29. 29.
    Besl, P., McKay, N.: A method for Registration of 3-D Shapes. IEEE Trans. PAMI 14, 239–256 (1992)CrossRefGoogle Scholar
  30. 30.
    Kazhdan, M., et al.: Poisson Surface Reconstruction. In: Proc. SGP, pp. 61–70 (2006)Google Scholar
  31. 31.
    Curless, B., Levoy, M.: A volumetric method for building complex models from range images. In: Proc. SIGGRAPH, pp. 303–312 (1996)Google Scholar
  32. 32.
    Furukawa, Y., Curless, B., Seitz, S.M., Szeliski, R.: Reconstructing Building Interiors from Images. In: Proc. ICCV (2009)Google Scholar
  33. 33.
    Min, D., Sohn, K.: Cost aggregation and occlusion handling with WLS in stereo matching. IEEE Trans. Image Proc., 1431–1442 (2008)Google Scholar
  34. 34.
    Hirschmüller, H.: Accurate and efficient stereo processing by semi-global matching and mutual information. IEEE Trans. PAMI 30, 328–341 (2008)CrossRefGoogle Scholar
  35. 35.
    Klaus, A., Sormann, M., Karner, K.: Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure. In: Proc. ICPR, pp. 15–18 (2006)Google Scholar
  36. 36.
    Kolmogorov, V., Zabih, R.: Computing visual correspondence with occlusions using graph cuts. In: Proc. ICCV (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Hansung Kim
    • 1
  • Adrian Hilton
    • 1
  1. 1.Centre for Vision, Speech and Signal ProcessingUniversity of SurreyGuildfordUK

Personalised recommendations