Computational Visual Media

, Volume 3, Issue 2, pp 95–106 | Cite as

Feature-based RGB-D camera pose optimization for real-time 3D reconstruction

Open Access
Research Article
  • 418 Downloads

Abstract

In this paper we present a novel feature-based RGB-D camera pose optimization algorithm for real-time 3D reconstruction systems. During camera pose estimation, current methods in online systems suffer from fast-scanned RGB-D data, or generate inaccurate relative transformations between consecutive frames. Our approach improves current methods by utilizing matched features across all frames and is robust for RGB-D data with large shifts in consecutive frames. We directly estimate camera pose for each frame by efficiently solving a quadratic minimization problem to maximize the consistency of 3D points in global space across frames corresponding to matched feature points. We have implemented our method within two state-of-the-art online 3D reconstruction platforms. Experimental results testify that our method is efficient and reliable in estimating camera poses for RGB-D data with large shifts.

Keywords

camera pose optimization feature matching real-time 3D reconstruction feature correspondence 

References

  1. [1]
    Keller, M.; Lefloch, D.; Lambers, M.; Izadi, S.; Weyrich, T.; Kolb, A. Real-time 3D reconstruction in dynamic scenes using point-based fusion. In: Proceedings of the International Conference on 3D Vision, 1–8, 2013.Google Scholar
  2. [2]
    Nießner, M.; Zollhöfer, M.; Izadi, S.; Stamminger, M. Real-time 3D reconstruction at scale using voxel hashing. ACM Transactions on Graphics Vol. 32, No. 6, Article No. 169, 2013.Google Scholar
  3. [3]
    Zhou, Q.-Y.; Koltun, V. Color map optimization for 3D reconstruction with consumer depth cameras. ACM Transactions on Graphics Vol. 33, No. 4, Article No. 155, 2014.Google Scholar
  4. [4]
    Newcombe, R. A.; Izadi, S.; Hilliges, O.; Molyneaux, D.; Kim, D.; Davison, A. J.; Kohi, P.; Shotton, J.; Hodges, S.; Fitzgibbon, A. KinectFusion: Real-time dense surface mapping and tracking. In: Proceedings of the 10th IEEE International Symposium on Mixed and Augmented Reality, 127–136, 2011.Google Scholar
  5. [5]
    Whelan, T.; Kaess, M.; Johannsson, H.; Fallon, M.; Leonard, J.; McDonald, J. Real-time large-scale dense RGB-D slam with volumetric fusion. The International Journal of Robotics Research Vol. 34, Nos. 4–5, 598–626, 2015.CrossRefGoogle Scholar
  6. [6]
    Peasley, B.; Birchfield, S. Replacing projective data association with Lucas–Kanade for KinectFusion. In: Proceedings of the IEEE International Conference on Robotics and Automation, 638–645, 2013.Google Scholar
  7. [7]
    Henry, P.; Krainin, M.; Herbst, E.; Ren, X.; Fox, D. RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments. The International Journal of Robotics Research Vol. 31, No. 5, 647–663, 2012.CrossRefGoogle Scholar
  8. [8]
    Newcombe, R. A.; Lovegrove, S. J.; Davison, A. J. DTAM: Dense tracking and mapping in real-time. In: Proceedings of the IEEE International Conference on Computer Vision, 2320–2327, 2011.Google Scholar
  9. [9]
    Whelan, T.; Leutenegger, S.; Salas-Moreno, R.; Glocker, B.; Davison, A. ElasticFusion: Dense SLAM without a pose graph. In: Proceedings of Robotics: Science and Systems, 11, 2015.Google Scholar
  10. [10]
    Whelan, T.; Johannsson, H.; Kaess, M.; Leonard, J. J.; McDonald, J. Robust real-time visual odometry for dense RGB-D mapping. In: Proceedings of the IEEE International Conference on Robotics and Automation, 5724–5731, 2013.Google Scholar
  11. [11]
    Zhang, K.; Zheng, S.; Yu, W.; Li, X. A depthincorporated 2D descriptor for robust and efficient 3D environment reconstruction. In: Proceedings of the 10th International Conference on Computer Science & Education, 691–696, 2015.Google Scholar
  12. [12]
    Endres, F.; Hess, J.; Engelhard, N.; Sturm, J.; Cremers, D.; Burgard, W. An evaluation of the RGB-D SLAM system. In: Proceedings of the IEEE International Conference on Robotics and Automation, 1691–1696, 2012.Google Scholar
  13. [13]
    Huang, A. S.; Bachrach, A.; Henry, P.; Krainin, M.; Maturana, D.; Fox, D.; Roy, N. Visual odometry and mapping for autonomous flight using an RGBD camera. In: Robotics Research. Christensen, H. I.; Khatib, O.; Eds. Springer International Publishing, 235–252, 2011.Google Scholar
  14. [14]
    Xiao, J.; Owens, A.; Torralba, A. SUN3D: A database of big spaces reconstructed using SfM and object labels. In: Proceedings of the IEEE International Conference on Computer Vision, 1625–1632, 2013.Google Scholar
  15. [15]
    Kümmerle, R.; Grisetti, G.; Strasdat, H.; Konolige, K.; Burgard, W. G2o: A general framework for graph optimization. In: Proceedings of the IEEE International Conference on Robotics and Automation, 3607–3613, 2011.Google Scholar
  16. [16]
    Roth, H.; Vona, M. Moving volume KinectFusion. In: Proceedings of British Machine Vision Conference, 1–11, 2012.Google Scholar
  17. [17]
    Zeng, M.; Zhao, F.; Zheng, J.; Liu, X. Octreebased fusion for realtime 3D reconstruction. Graphical Models Vol. 75, No. 3, 126–136, 2013.CrossRefGoogle Scholar
  18. [18]
    Whelan, T.; Johannsson, H.; Kaess, M.; Leonard, J. J.; McDonald, J. Robust tracking for real-time dense RGB-D mapping with Kintinuous. Computer Science and Artificial Intelligence Laboratory Technical Report, MIT-CSAIL-TR-2012-031, 2012.Google Scholar
  19. [19]
    Henry, P.; Fox, D.; Bhowmik, A.; Mongia, R. Patch volumes: Segmentation-based consistent mapping with RGBD cameras. In: Proceedings of the International Conference on 3D Vision, 398–405, 2013.Google Scholar
  20. [20]
    Chen, J.; Bautembach, D.; Izadi, S. Scalable real-time volumetric surface reconstruction. ACM Transactions on Graphics Vol. 32, No. 4, Article No. 113, 2013.Google Scholar
  21. [21]
    Steinbrucker, F.; Kerl, C.; Cremers, D. Large-scale multiresolution surface reconstruction from RGB-D sequences. In: Proceedings of the IEEE International Conference on Computer Vision, 3264–3271, 2013.Google Scholar
  22. [22]
    Sturm, J.; Engelhard, N.; Endres, F.; Burgard, W.; Cremers, D. A benchmark for the evaluation of RGBD SLAM systems. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 573–580, 2012.Google Scholar
  23. [23]
    Stückler, J.; Behnke, S. Multi-resolution surfel maps for efficient dense 3D modeling and tracking. Journal of Visual Communication and Image Representation Vol. 25, No. 1, 137–147, 2014.CrossRefGoogle Scholar
  24. [24]
    Zhou, Q.-Y.; Koltun, V. Dense scene reconstruction with points of interest. ACM Transactions on Graphics Vol. 32, No. 4, Article No. 112, 2013.Google Scholar
  25. [25]
    Zhou, Q.-Y.; Miller, S.; Koltun, V. Elastic fragments for dense scene reconstruction. In: Proceedings of the IEEE International Conference on Computer Vision, 473–480, 2013Google Scholar
  26. [26]
    Choi, S.; Zhou, Q.-Y.; Koltun, V. Robust reconstruction of indoor scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5556–5565, 2015.Google Scholar
  27. [27]
    Steinbrücker, F.; Sturm, J.; Cremers, D. Realtime visual odometry from dense RGB-D images. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, 719–722, 2011.Google Scholar
  28. [28]
    H¨ansch, R.; Weber, T.; Hellwich, O. Comparison of 3D interest point detectors and descriptors for point cloud fusion. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences Vol. 2, No. 3, 57, 2014.CrossRefGoogle Scholar
  29. [29]
    Juan, L.; Gwun, O. A comparison of SIFT, PCA-SIFT and SURF. International Journal of Image Processing Vol. 3, No. 4, 143–152, 2009.Google Scholar
  30. [30]
    Yan, W.; Shi, X.; Yan, X.; Wan, L. Computing openSURF on openCL and general purpose GPU. International Journal of Advanced Robotic Systems Vol. 10, No. 10, 375, 2013.CrossRefGoogle Scholar
  31. [31]
    Hartley, R.; Zisserman, A. Multiple View Geometry in Computer Vision. Cambridge University Press, 2003.MATHGoogle Scholar
  32. [32]
    Arun, K. S.; Huang, T. S.; Blostein, S. D. Leastsquares fitting of two 3-D point sets. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. PAMI-9, No. 5, 698–700, 1987.CrossRefGoogle Scholar

Copyright information

© The Author(s) 2016

Open Access The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.University of Texas at DallasRichardsonUSA

Personalised recommendations