Elaborate Scene Reconstruction with a Consumer Depth Camera

Research Article

Abstract

A robust approach to elaborately reconstruct the indoor scene with a consumer depth camera is proposed in this paper. In order to ensure the accuracy and completeness of 3D scene model reconstructed from a freely moving camera, this paper proposes new 3D reconstruction methods, as follows: 1) Depth images are processed with a depth adaptive bilateral filter to effectively improve the image quality; 2) A local-to-global registration with the content-based segmentation is performed, which is more reliable and robust to reduce the visual odometry drifts and registration errors; 3) An adaptive weighted volumetric method is used to fuse the registered data into a global model with sufficient geometrical details. Experimental results demonstrate that our approach increases the robustness and accuracy of the geometric models which were reconstructed from a consumer-grade depth camera.

Keywords

3D reconstruction image processing geometry registration simultaneous localization and mapping (SLAM) volumetric integration 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    S. Izadi, D. Kim, O. Hilliges, D. Molyneaux, R. Newcombe, P. Kohli, J. Shotton, S. Hodges, D. Freeman, A. Davison, A. Fitzgibbon. KinectFusion: Real-time 3D reconstruction and interaction using a moving depth camera. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, ACM, Santa Barbara, USA, pp. 559–568, 2011.Google Scholar
  2. [2]
    R. A. Newcombe, S. Izadi, O. Hilliges, D. Molyneaux, D. Kim, A. J. Davison, P. Kohi, J. Shotton, S. Hodges, A. Fitzgibbon. KinectFusion: Real-time dense surface mapping and tracking. In Proceedings of the 10th IEEE International Symposium on Mixed and Augmented Reality, IEEE, Basel, Switzerland, pp. 127–136, 2011.Google Scholar
  3. [3]
    B. Curless, M. Levoy. A volumetric method for building complex models from range images. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, ACM, New York, USA, pp. 303–312, 1996.Google Scholar
  4. [4]
    S. Rusinkiewicz, M. Levoy. Efficient variants of the ICP algorithm. In Proceedings of the 3rd International Conference on 3D Digital Imaging and Modeling, IEEE, Quebec, Canada, pp. 145–152, 2001.CrossRefGoogle Scholar
  5. [5]
    C. Kerl, J. Sturm, D. Cremers. Robust odometry estimation for RGB-D cameras. In Proceedings of IEEE International Conference on Robotics and Automation, IEEE, Karlsruhe, Germany, pp. 3748–3754, 2013.Google Scholar
  6. [6]
    F. Steinbrücker, J. Sturm, D. Cremers. Realtime visual odometry from dense RGB-D images. In Proceedings of IEEE International Conference on Computer Vision Workshops, IEEE, Barcelona, Spain, pp. 719–722, 2011.Google Scholar
  7. [7]
    T. Whelan, M. Kaess, H. Johannsson, M. Fallon, J. J. Leonard, J. McDonald. Real-time large-scale dense RGBD SLAM with volumetric fusion. International Journal of Robotics Research, vol. 34, no. 4–5, pp. 598–626, 2015.CrossRefGoogle Scholar
  8. [8]
    J. Huai, Y. Zhang, A. Yilmaz. Real-time large scale 3D reconstruction by fusing Kinect and IMU data. In Proceedings of ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, ISPRS, La Grande Motte, France, vol. II-3/W5, pp. 491–496, 2015.Google Scholar
  9. [9]
    M. Niessner, A. Dai, M. Fisher. Combining inertial navigation and ICP for real-time 3D surface reconstruction. Eurographics, E. Galin, M. Wand, Eds., Strasbourg, France: The Eurographics Association, pp. 13–16, 2014.Google Scholar
  10. [10]
    K. H. Yang, W. S. Yu, X. Q. Ji. Rotation estimation for mobile robot based on Single-axis gyroscope and monocular camera. International Journal of Automation and Computing, vol. 9, no. 3, pp. 292–298, 2012.CrossRefGoogle Scholar
  11. [11]
    S. Choi, Q. Y. Zhou, V. Koltun. Robust reconstruction of indoor scenes. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Boston, USA, pp. 5556–5565, 2015.Google Scholar
  12. [12]
    T. Whelan, M. Kaess, J. J. Leonard, J. McDonald. Deformation-based loop closure for large scale dense RGBD SLAM. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, Tokyo, Japan, pp. 548–555, 2013.Google Scholar
  13. [13]
    Q. Y. Zhou, S. Miller, V. Koltun. Elastic fragments for dense scene reconstruction. In Proceedings of IEEE International Conference on Computer Vision, IEEE, Sydney, Australia, pp. 473–480, 2013.Google Scholar
  14. [14]
    A. Chatterjee, V. M. Govindu. Noise in structured-light stereo depth cameras: Modeling and its applications. arXiv: 1505.01936, 2015.Google Scholar
  15. [15]
    K. Khoshelham. Accuracy analysis of Kinect depth data. In Proceedings of International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, ISPRS, Calgary, Canada, vol. XXXVIII-5/W12, pp. 133–138, 2011.Google Scholar
  16. [16]
    K. Khoshelham, S. O. Elberink. Accuracy and resolution of Kinect depth data for indoor mapping applications. Sensors, vol. 12, no. 2, pp. 1437–1454, 2012.CrossRefGoogle Scholar
  17. [17]
    C. V. Nguyen, S. Izadi, D. Lovell. Modeling Kinect sensor noise for improved 3D reconstruction and tracking. In Proceedings of the 2nd International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission, IEEE, Zurich, Switzerland, pp. 524–530, 2012.Google Scholar
  18. [18]
    J. Smisek, M. Jancosek, T. Pajdla. 3D with Kinect. In Proceedings of IEEE International Conference on Computer Vision Workshops, IEEE, Barcelona, Spain, pp. 1154–1160, 2011.Google Scholar
  19. [19]
    C. Tomasi, R. Manduchi. Bilateral filtering for gray and color images. In Proceedings of the 6th International Conference on Computer Vision, IEEE, Bombay, India, pp. 839–846, 1998.Google Scholar
  20. [20]
    J. X. Xiao, A. Owens, A. Torralba. SUN3D: A database of big spaces reconstructed using SFM and object labels. In Proceedings of IEEE International Conference on Computer Vision, IEEE, Sydney, Australia, pp. 1625–1632, 2013.Google Scholar
  21. [21]
    Q. F. Chen, V. Koltun. Fast MRF optimization with application to depth reconstruction. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Columbus, USA, pp. 3914–3921, 2014.Google Scholar
  22. [22]
    J. Kopf, M. F. Cohen, D. Lischinski, M. Uyttendaele. Joint bilateral upsampling. ACM Transactions on Graphics, vol. 26, no. 3, Article number 96, 2007.CrossRefGoogle Scholar
  23. [23]
    T. Whelan, M. Kaess, M. Fallon, H. Johannsson, J. Leonard, J. McDonald. Kintinuous: Spatially extended KinectFusion. In Proceedings of RSS Workshop on RGB-D: Advanced Reasoning with Depth Cameras, Sydney, Australia, pp. 3–14, 2012.Google Scholar
  24. [24]
    A. Zeng, S. Song, M. Niessner, M. Fisher, J. X. Xiao, T. Funkhouser. 3DMatch: Learning the matching of local 3D geometry in range scans. In Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Puerto Rico, USA, 2017.Google Scholar
  25. [25]
    M. Halber, T. Funkhouser. Structured global registration of RGB-D scans in indoor environments. arXiv:1607.08539, 2016.Google Scholar
  26. [26]
    S. Parker, P. Shirley, Y. Livnat, C. Hansen, P. P. Sloan. Interactive ray tracing for isosurface rendering. In Proceedings of the Conference on Visualization, IEEE, North Carolina, USA, pp. 233–238, 1998.Google Scholar
  27. [27]
    W. E. Lorensen, H. E. Cline. Marching cubes: A high resolution 3D surface construction algorithm. ACM SIGGRAPH Computer Graphics, vol. 21, no. 4, pp. 163–169, 1987.CrossRefGoogle Scholar
  28. [28]
    M. Zollhöfer, A. Dai, M. Innmann, C. L. Wu, M. Stamminger, C. Theobalt, M. Niessner. Shading-based refinement on volumetric signed distance functions. ACM Transactions on Graphics, vol. 34, no. 4, Article number 96, 2015.CrossRefMATHGoogle Scholar
  29. [29]
    Q. Y. Zhou, V. Koltun. Dense scene reconstruction with points of interest. ACM Transactions on Graphics, vol. 32, no. 4, Article number 112, 2013.MATHGoogle Scholar
  30. [30]
    Q. Y. Zhou, J. Park, V. Koltun. Fast global registration. In Proceedings of 14th European Conference on Computer Vision, Amsterdam, The Netherlands, pp. 766–782, 2016.Google Scholar
  31. [31]
    R Kümmerle, G. Grisetti, H. Strasdat, K. Konolige, W. Burgard. g2o: A general framework for graph optimization. In Proceedings of IEEE International Conference on Robotics and Automation, IEEE, Shanghai, China, pp. 3607–3613, 2011.Google Scholar
  32. [32]
    T. Whelan, S. Leutenegger, R. F. Salas-Moreno, B. Glocker, A. J. Davison. ElasticFusion: Dense SLAM without a pose graph. In Proceedings of Robotics: Science and Systems, Rome, Italy, 2015.Google Scholar
  33. [33]
    A. Handa, T. Whelan, J. McDonald, A. J. Davison. A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM. In Proceedings of IEEE International Conference on Robotics and Automation, IEEE, Hong Kong, China, pp. 1524–1531, 2014.Google Scholar
  34. [34]
    C. Kerl, J. Sturm, D. Cremers. Dense visual SLAM for RGB-D cameras. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, Tokyo, Japan, pp. 2100–2106, 2013.Google Scholar

Copyright information

© Institute of Automation, Chinese Academy of Sciences and Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.National Laboratory of Pattern Recognition, Institute of AutomationChinese Academy of SciencesBeijingChina
  2. 2.University of Chinese Academy of SciencesBeijingChina

Personalised recommendations