Advertisement

Color-Introduced Frame-to-Model Registration for 3D Reconstruction

  • Fei LiEmail author
  • Yunfan Du
  • Rujie Liu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10133)

Abstract

3D reconstruction has become an active research topic with the popularity of consumer-grade RGB-D cameras, and registration for model alignment is one of the most important steps. Most typical systems adopt depth-based geometry matching, while the captured color images are totally discarded. Some recent methods further introduce photometric cue for better results, but only frame-to-frame matching is used. In this paper, a novel registration approach is proposed. According to both geometric and photometric consistency, depth and color information are involved in a unified optimization framework. With the available depth maps and color images, a global model with colored surface vertices is maintained. The incoming RGB-D frames are aligned based on frame-to-model matching for more effective camera pose estimation. Both quantitative and qualitative experimental results demonstrate that better reconstruction performance can be obtained by our proposal.

Keywords

3D reconstruction Color mapping Registration Frame-to-model matching Optimization 

References

  1. 1.
    Smisek, J., Jancosek, M., Pajdla, T.: 3D with kinect. In: Fossati, A., Gall, J., Grabner, H., Ren, X., Konolige, K. (eds.) Consumer Depth Cameras for Computer Vision, pp. 3–25. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  2. 2.
    Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., Fitzgibbon, A.: KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera. In: Proceedings of ACM Symposium on User Interface Software and Technology, pp. 559–568 (2011)Google Scholar
  3. 3.
    Newcombe, R.A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A.J., Kohli, P., Shotton, J., Hodges, S., Fitzgibbon, A.: KinectFusion: real-time dense surface mapping and tracking. In: Proceedings of IEEE International Symposium on Mixed and Augmented Reality, pp. 127–136 (2011)Google Scholar
  4. 4.
    Zhou, Q.Y., Koltun, V.: Dense scene reconstruction with points of interest. ACM Trans. Graph. 32 (2013)Google Scholar
  5. 5.
    Zhou, Q.Y., Miller, S., Koltun, V.: Elastic fragments for dense scene reconstruction. In: Proceedings of IEEE International Conference on Computer Vision, pp. 473–480 (2013)Google Scholar
  6. 6.
    Nießner, M., Zollhöfer, M., Izadi, S., Stamminger, M.: Real-time 3D reconstruction at scale using voxel hashing. ACM Trans. Graph. 32 (2013)Google Scholar
  7. 7.
    Zhou, Q.Y., Koltun, V.: Simultaneous localization and calibration: self-calibration of consumer depth cameras. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, pp. 454–460 (2014)Google Scholar
  8. 8.
    Choe, G., Park, J., Tai, Y.W., Kweon, I.S.: Exploiting shading cues in kinect IR images for geometry refinement. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, pp. 3922–3929 (2014)Google Scholar
  9. 9.
    Zollhöfer, M., Nießner, M., Izadi, S., Rhemann, C., Zach, C., Fisher, M., Wu, C., Fitzgibbon, A., Loop, C., Theobalt, C., Stamminger, M.: Real-time non-rigid reconstruction using an RGB-D camera. ACM Trans. Graph. 33 (2014)Google Scholar
  10. 10.
    Newcombe, R.A., Fox, D., Seitz, S.M.: DynamicFusion: reconstruction and tracking of non-rigid scenes in real-time. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, pp. 343–352 (2015)Google Scholar
  11. 11.
    Dou, M., Taylor, J., Fuchs, H., Fitzgibbon, A., Izadi, S.: 3D scanning deformable objects with a single RGBD sensor. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, pp. 493–501 (2015)Google Scholar
  12. 12.
    Innmann, M., Zollhöfer, M., Nießner, M., Theobalt, C., Stamminger, M.: VolumeDeform: real-time volumetric non-rigid reconstruction. arXiv preprint arXiv:1603.08161 (2016)
  13. 13.
    Rusinkiewicz, S., Hall-Holt, O., Levoy, M.: Real-time 3D model acquisition. ACM Trans. Graph. 21, 438–446 (2002)CrossRefGoogle Scholar
  14. 14.
    Herrera, D., Kannala, J., Heikkilä, J.: Joint depth and color camera calibration with distortion correction. IEEE Trans. Pattern Anal. Mach. Intell. 34, 2058–2064 (2012)CrossRefGoogle Scholar
  15. 15.
    Teichman, A., Miller, S., Thrun, S.: Unsupervised intrinsic calibration of depth sensors via SLAM. In: Robotics: Science and Systems (2013)Google Scholar
  16. 16.
    Kerl, C., Sturm, J., Cremers, D.: Dense visual SLAM for RGB-D cameras. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2100–2106 (2013)Google Scholar
  17. 17.
    Kerl, C., Sturm, J., Cremers, D.: Robust odometry estimation for RGB-D cameras. In: Proceedings of IEEE International Conference on Robotics and Automation, pp. 3748–3754 (2013)Google Scholar
  18. 18.
    Whelan, T., Johannsson, H., Kaess, M., Leonard, J.J., McDonald, J.: Robust real-time visual odometry for dense RGB-D mapping. In: Proceedings of IEEE International Conference on Robotics and Automation, pp. 5724–5731 (2013)Google Scholar
  19. 19.
    Wang, K., Zhang, G., Bao, H.: Robust 3D reconstruction with an RGB-D camera. IEEE Trans. Image Process. 23, 4893–4906 (2014)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Choi, S., Zhou, Q.Y., Miller, S., Koltun, V.: A large dataset of object scans. arXiv preprint arXiv:1602.02481 (2016)
  21. 21.
    Curless, B., Levoy, M.: A volumetric method for building complex models from range images. ACM Trans. Graph. 303–312 (1996)Google Scholar
  22. 22.
    Parker, S., Shirley, P., Livnat, Y., Hansen, C., Sloan, P.P.: Interactive ray tracing for isosurface rendering. In: Proceedings of IEEE Conference on Visualization, pp. 233–238 (1998)Google Scholar
  23. 23.
    Zhou, Q.Y., Koltun, V.: Color map optimization for 3D reconstruction with consumer depth cameras. ACM Trans. Graph. 33 (2014)Google Scholar
  24. 24.
    Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-D SLAM systems. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 573–580 (2012)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Fujitsu Research and Development Center Co., Ltd.BeijingChina

Personalised recommendations