Skip to main content
SpringerLink
Log in

Efficient multi-plane extraction from massive 3D points for modeling large-scale urban scenes

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In modeling large-scale urban scenes, extracting reliable dominant planes from initial 3D points plays an important role for inferring the complete scene structures. However, traditional local and global methods are frequently prone to missing many real planes and also appear powerless when massive 3D points are present. To solve these problems, the paper presents an efficient multi-plane extraction method based on scene structure priors. The proposed method first explores the potential relations between the planes by detecting 2D line segments in the projection map produced from initial 3D points (i.e., simplify 3D model to 2D model), including: (1) multi-line detection in regions by the guidance of scene structure priors; (2) multi-line detection between regions under the Markov Random Field framework incorporating scene structure priors. Then, according to the resulting plane relations, a rapid multi-plane generation is carried out instead of the time-consuming plane fitting over 3D points. Experimental results confirm that the proposed method can efficiently produce sufficient and reliable dominant planes from a vast number of noisy 3D points (only about 8 s on 2000K 3D points) and can be applied for modeling large-scale urban scenes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Mičušík, B., Košecká, J.: Multi-view superpixel stereo in urban environments. Int. J. Comput. Vis. 89(1), 106–119 (2010)

    Article  Google Scholar 

  2. Kim, H., Xiao, H., Max, N.: Piecewise planar scene reconstruction and optimization for multi-view stereo. In: Proceedings of Asian Conference on Computer Vision, pp. 191–204 (2012)

  3. Bodis-Szomoru, A., Riemenschneider, H., Van Gool, L.: Fast, approximate piecewise-planar modeling based on sparse structure-from-motion and superpixel. In: Proceedings of Computer Vision and Pattern Recognition, pp. 469–476 (2014)

  4. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  5. Zuliani, M., Kenney, C.S., Manjunath, B.S.: The multiransac algorithm and its application to detect planar homographies. In: Proceedings of the IEEE International Conference on Image Processing, pp. III-153–6 (2005)

  6. Vincent, T., Laganiere, R.: Detecting planar homographies in an image pair. In: Image and Signal Processing and Analysis, pp. 182–187 (2001)

  7. Toldo, R., Fusiello, A.: Robust multiple structures estimation with J-linkage. In: Proceedings of the European Conference on Computer Vision, pp. 537–547 (2008)

  8. Chin, T.J., Yu, J., Suter, D.: Accelerated hypothesis generation for multi-structure data via preference analysis. IEEE Trans. Pattern Anal. Mach. Intell. 34(4), 625–638 (2012)

    Article  Google Scholar 

  9. Pham, T.T., Chin, T.J., Yu, J., Suter, D.: Simultaneous sampling and multi-structure fitting with adaptive reversible jump MCMC. In: Advances in Neural Information Processing Systems (NIPS), pp. 540–548 (2011)

  10. Thakoor, N., Gao, J.: Branch-and-bound hypothesis selection for two-view multiple structure and motion segmentation. In: Proceedings of Computer Vision and Pattern Recognition, pp. 1–6 (2008)

  11. Lazic, N., Givoni, I., Frey, B., Aarabi, P.: FLoSS: facility location for subspace segmentation. In: Proceedings of IEEE International Conference on Computer Vision, pp. 825–832 (2009)

  12. Isack, H., Boykov, Y.: Energy-based geometric multi-model fitting. Int. J. Comput. Vis. 97(2), 123–147 (2010)

    Article  MATH  Google Scholar 

  13. Delong, A., Osokin, A., Isack, H.N., et al.: Fast approximate energy minimization with label costs. Int. J. Comput. Vis. 96(1), 1–27 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  14. Yu, J., Chin, T.J., Suter, D.: A global optimization approach to robust multi-model fitting. In: Proceedings of Computer Vision and Pattern Recognition, pp. 2041–2048 (2011)

  15. Pham, T.T., Chin, T.J., Yu, J., Suter, D.: The random cluster model for robust geometric fitting. IEEE Trans. Pattern Anal. Mach. Intell. 36(8), 1658–1671 (2014)

    Article  Google Scholar 

  16. Furukawa, Y., Curless, B., Seitz, S.M., Szeliski, R.: Manhattan-world stereo. In: Proceedings of Computer Vision and Pattern Recognition, pp. 1422–1429 (2010)

  17. Sinha, S.N., Steedly, D., Szeliski, R.: Piecewise planar stereo for image-based rendering. In: Proceedings of IEEE International Conference on Computer Vision, pp. 1881–1888 (2009)

  18. Chauve, A., Labatut, P., Pons, J.: Robust piecewise-planar 3D reconstruction and completion from large-scale unstructured point data. In: Proceedings of Computer Vision and Pattern Recognition, pp. 1261–1268 (2010)

  19. Verleysen, C., Vleeschouwer, C.D.: Piecewise-planar 3D approximation from wide-baseline stereo. In: Proceedings of Computer Vision and Pattern Recognition, pp. 3327–3336 (2016)

  20. Monszpart, A., Mellado, N., Brostow, J.G., Mitra, J.N.: RAPter: rebuilding man-made scenes with regular arrangements of planes. ACM SIGGRAPH 34(4), 103:1–103:12 (2015)

    Article  Google Scholar 

  21. Wang, J., Xu, K.: Shape detection from raw LiDAR data with subspace modeling. IEEE Trans. Vis. Comput. Graph. 99, 1–14 (2016)

    Google Scholar 

  22. Tsampikos Michailidis, G., Pajarola, R.: Bayesian graph-cut optimization for wall surfaces reconstruction in indoor environments. Vis. Comput. 33, 1347–1355 (2017)

    Article  Google Scholar 

  23. Wolberg, G., Zokai, S.: PhotoSketch: a photocentric urban 3D modeling system. Vis. Comput. (2017). https://doi.org/10.1007/s00371-017-1365-x

  24. Furukawa, Y., Ponce, J.: Accurate, dense, and robust multi-view stereopsis. IEEE Trans. Pattern Anal. Mach. Intell. 32(8), 1362–1376 (2009)

    Article  Google Scholar 

  25. Delong, A., Osokin, A., Isack, H., Boykov, Y.: Fast approximate energy minimization with label costs. Int. J. Comput. Vis. 96(1), 1–27 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  26. http://vision.ia.ac.cn/data/index.html. Accessed 23 Dec 2017

Download references

Acknowledgements

This work is supported in part by the National Key R&D Program of China (2016YFB0502002), and in part by the Open Project Program of the National Laboratory of Pattern Recognition (201700004), the National Natural Science Foundation of China (61472419), the Natural Science Foundation of Henan Province (162300410347), the College Key Research Project of Henan Province (17A520018, 17A520019), the School-Based Project of Zhoukou Normal University (zknuc2015103, zknub2201705)

Author information

Authors and Affiliations

  1. School of Network Engineering, Zhoukou Normal University, Zhoukou, 466000, China

    Wei Wang

  2. Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China

    Wei Gao

  3. University of Chinese Academy of Sciences, Beijing, 100049, China

    Wei Gao

Authors
  1. Wei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, W., Gao, W. Efficient multi-plane extraction from massive 3D points for modeling large-scale urban scenes. Vis Comput 35, 625–638 (2019). https://doi.org/10.1007/s00371-018-1492-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-018-1492-z

Keywords

Search

Navigation