Skip to main content
SpringerLink
Log in
Book cover

Consumer Depth Cameras for Computer Vision pp 27–48Cite as

Real-Time RGB-D Mapping and 3-D Modeling on the GPU Using the Random Ball Cover

  • Chapter

Part of the book series: Advances in Computer Vision and Pattern Recognition ((ACVPR))

Abstract

In this chapter, we present a system for real-time point cloud mapping and scene reconstruction based on an efficient implementation of the iterative closest point (ICP) algorithm on the graphics processing unit (GPU). Compared to state-of-the-art approaches that achieve real-time performance using projective data association schemes which operate on the 3-D scene geometry solely, our method allows to incorporate additional complementary information to guide the registration process. In this work, the ICP’s nearest neighbor search evaluates both geometric and photometric information in a direct manner, achieving robust mappings in real-time. In order to overcome the performance bottleneck in nearest neighbor search space traversal, we exploit the inherent computation parallelism of GPUs. In particular, we have adapted the random ball cover (RBC) data structure and search algorithm, originally proposed for high-dimensional problems, to low-dimensional RGB-D data. The system is validated on scene and object reconstruction scenarios. Our implementation achieves frame-to-frame registration runtimes of less than 20 ms on an off-the-shelf consumer GPU.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Akenine-Möller, T., Haines, E., Hoffman, N.: Real-Time Rendering, 3rd edn. AK Peters, Natick, MA (2008)

    Book  Google Scholar 

  2. Bailey, T., Durrant-Whyte, H.: Simultaneous localization and mapping (SLAM): part II. IEEE Robot. Autom. Mag. 13(3), 108–117 (2006)

    Article  Google Scholar 

  3. Bauer, S., Wasza, J., Haase, S., Marosi, N., Hornegger, J.: Multi-modal surface registration for markerless initial patient setup in radiation therapy using Microsoft’s Kinect sensor. In: International Conference on Computer Vision—Workshop on Consumer Depth Cameras for Computer Vision, pp. 1175–1181 (2011)

    Google Scholar 

  4. Besl, P., McKay, N.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)

    Article  Google Scholar 

  5. Blais, G., Levine, D.M.: Registering multiview range data to create 3-D computer objects. IEEE Trans. Pattern Anal. Mach. Intell. 17(8), 820–824 (1995)

    Article  Google Scholar 

  6. Castaneda, V., Mateus, D., Navab, N.: SLAM combining ToF and high-resolution cameras. In: IEEE Workshop on Applications of Computer Vision, pp. 672–678 (2011)

    Chapter  Google Scholar 

  7. Cayton, L.: A nearest neighbor data structure for graphics hardware. In: International Workshop on Accelerating Data Management Systems Using Modern Processor and Storage Architectures (2010)

    Google Scholar 

  8. Cayton, L.: Accelerating nearest neighbor search on manycore systems. CoRR arXiv:1103.2635 (2011)

  9. Chen, Y., Medioni, G.: Object modelling by registration of multiple range images. Image Vis. Comput. 10(3), 145–155 (1992)

    Article  Google Scholar 

  10. Curless, B., Levoy, M.: A volumetric method for building complex models from range images. In: Conference on Computer Graphics and Interactive Techniques, SIGGRAPH, pp. 303–312. ACM, New York (1996)

    Google Scholar 

  11. Druon, S., Aldon, M., Crosnier, A.: Color constrained ICP for registration of large unstructured 3D color data sets. In: IEEE International Conference on Information Acquisition, pp. 249–255 (2006)

    Chapter  Google Scholar 

  12. Engelhard, N., Endres, F., Hess, J., Sturm, J., Burgard, W.: Real-time 3D visual SLAM with a hand-held RGB-D camera. In: RGB-D Workshop on 3D Perception in Robotics, European Robotics Forum (2011)

    Google Scholar 

  13. Fioraio, N., Konolige, K.: Realtime visual and point cloud SLAM. In: RGB-D Workshop: Advanced Reasoning with Depth Cameras, Robotics Science and Systems Conference (2011)

    Google Scholar 

  14. Garcia, J., Zalevsky, Z.: Range mapping using speckle decorrelation. US Patent No. 7433024 (2008)

    Google Scholar 

  15. Garcia, V., Debreuve, E., Barlaud, M.: Fast k nearest neighbor search using GPU. In: IEEE Conference on Computer Vision and Pattern Recognition—Workshop on Computer Vision on GPU (2008)

    Google Scholar 

  16. Gevers, T., Smeulders, A.W.: Color-based object recognition. Pattern Recognit. 32(3), 453–464 (1999)

    Article  Google Scholar 

  17. Harris, M., Sengupta, S., Owens, J.D.: Parallel prefix sum (scan) with CUDA. In: GPU Gems 3, pp. 851–876. Addison-Wesley, Reading (2007)

    Google Scholar 

  18. He, K., Sun, J., Tang, X.: Guided image filtering. In: European Conference on Computer Vision, pp. 1–14 (2010)

    Google Scholar 

  19. Henry, P., Krainin, M., Herbst, E., Ren, X., Fox, D.: RGB-D mapping: using depth cameras for dense 3D modeling of indoor environments. In: International Symposium on Experimental Robotics (2010)

    Google Scholar 

  20. Hoberock, J., Bell, N.: Thrust: a parallel template library (2010). URL http://code.google.com/p/thrust/. Version 1.3.0

  21. Horn, B.: Closed-form solution of absolute orientation using unit quaternions. J. Opt. Soc. Am. 4(4), 629–642 (1987)

    Article  MathSciNet  Google Scholar 

  22. Huhle, B., Jenke, P., Strasser, W.: On-the-fly scene acquisition with a handy multi-sensor system. Int. J. Intell. Syst. Technol. Appl. 5, 255–263 (2008)

    Google Scholar 

  23. Izadi, S., Newcombe, R.A., Kim, D., Hilliges, O., Molyneaux, D., Hodges, S., Kohli, P., Shotton, J., Davison, A.J., Fitzgibbon, A.W.: KinectFusion: real-time dynamic 3D surface reconstruction and interaction. In: ACM Symposium on User Interface Software and Technology, p. 23 (2011)

    Google Scholar 

  24. Johnson, A., Kang, S.B.: Registration and integration of textured 3-D data. In: International Conference on Recent Advances in 3-D Digital Imaging and Modeling, pp. 234–241 (1997)

    Google Scholar 

  25. Joung, J.H., An, K.H., Kang, J.W., Chung, M.J., Yu, W.: 3D environment reconstruction using modified color ICP algorithm by fusion of a camera and a 3D laser range finder. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3082–3088 (2009)

    Google Scholar 

  26. Korn, G.A., Korn, T.M.: Mathematical Handbook for Scientists and Engineers: Definitions, Theorems, and Formulas for Reference and Review. Dover, New York (2000)

    Google Scholar 

  27. Loop, C., Blinn, J.: Real-time GPU rendering of piecewise algebraic surfaces. ACM Trans. Graph. 25(3), 664–670 (2006)

    Article  Google Scholar 

  28. May, S., Droeschel, D., Holz, D., Fuchs, S., Malis, E., Nüchter, A., Hertzberg, J.: Three-dimensional mapping with time-of-flight cameras. J. Field Robot. 26, 934–965 (2009)

    Article  Google Scholar 

  29. McGuire, M.: A fast, small-radius GPU median filter. In: ShaderX6, pp. 165–173. Charles River Media (2008)

    Google Scholar 

  30. Neumann, D., Lugauer, F., Bauer, S., Wasza, J., Hornegger, J.: Real-time RGB-D mapping and 3-D modeling on the GPU using the random ball cover data structure. In: International Conference on Computer Vision—Workshop on Consumer Depth Cameras for Computer Vision, pp. 1161–1167 (2011)

    Google Scholar 

  31. Newcombe, R.A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A.J., Kohli, P., Shotton, J., Hodges, S., Fitzgibbon, A.W.: KinectFusion: real-time dense surface mapping and tracking. In: IEEE International Symposium on Mixed and Augmented Reality, pp. 127–136 (2011)

    Chapter  Google Scholar 

  32. Nüchter, A., Surmann, H., Lingemann, K., Hertzberg, J., Thrun, S.: 6D SLAM with an application in autonomous mine mapping. In: IEEE International Conference on Robotics and Automation, vol. 2, pp. 1998–2003 (2004)

    Google Scholar 

  33. Qiu, D., May, S., Nüchter, A.: GPU-accelerated nearest neighbor search for 3D registration. In: International Conference on Computer Vision Systems, pp. 194–203. Springer, Berlin (2009)

    Chapter  Google Scholar 

  34. Reis, G., Zeilfelder, F., Hering-Bertram, M., Farin, G.E., Hagen, H.: High-quality rendering of quartic spline surfaces on the GPU. IEEE Trans. Vis. Comput. Graph. 14(5), 1126–1139 (2008)

    Article  Google Scholar 

  35. Rusinkiewicz, S., Hall-Holt, O., Levoy, M.: Real-time 3D model acquisition. ACM Trans. Graph. 21(3), 438–446 (2002)

    Article  Google Scholar 

  36. Rusinkiewicz, S., Levoy, M.: Efficient variants of the ICP algorithm. In: International Conference on 3-D Digital Imaging and Modeling, pp. 145–152 (2001)

    Chapter  Google Scholar 

  37. Wasza, J., Bauer, S., Haase, S., Hornegger, J.: Real-time preprocessing for dense 3-D range imaging on the GPU: defect interpolation, bilateral temporal averaging and guided filtering. In: International Conference on Computer Vision—Workshop on Consumer Depth Cameras for Computer Vision, pp. 1221–1227 (2011)

    Google Scholar 

Download references

Acknowledgements

S. Bauer and J. Wasza gratefully acknowledge the support by the European Regional Development Fund (ERDF) and the Bayerisches Staatsministerium für Wirtschaft, Infrastruktur, Verkehr und Technologie (StMWIVT), in the context of the R&D program IuK Bayern under Grant No. IUK338. Furthermore, this research was supported by the Graduate School of Information Science in Health (GSISH) and the TUM Graduate School.

Author information

Authors and Affiliations

  1. Pattern Recognition Lab, Department of Computer Science, Friedrich-Alexander-Universität Erlangen-Nürnberg, Martensstr. 3, 91058, Erlangen, Germany

    Sebastian Bauer, Jakob Wasza, Felix Lugauer & Dominik Neumann

  2. Erlangen Graduate School in Advanced Optical Technologies (SAOT) & Pattern Recognition Lab, Department of Computer Science, Friedrich-Alexander-Universität Erlangen-Nürnberg, Martensstr. 3, 91058, Erlangen, Germany

    Joachim Hornegger

Authors
  1. Sebastian Bauer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jakob Wasza
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Felix Lugauer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dominik Neumann
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Joachim Hornegger
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sebastian Bauer .

Editor information

Editors and Affiliations

  1. Computer Vision Laboratory, ETH Zürich, Sternwartstrasse 7, Zürich, 8092, Switzerland

    Andrea Fossati

  2. Perceiving Systems Department, Max Planck Inst. for Intelligent Systems, Spemannstrasse 41, Tübingen, 72076, Germany

    Juergen Gall

  3. Computer Vision Laboratory, ETH Zürich, Sternwartstrasse 7, Zürich, 8092, Switzerland

    Helmut Grabner

  4. Intel Science and Technology Center, Allen Center 462, Seattle, 98195, Washington, USA

    Xiaofeng Ren

  5. Industrial Perception, Industrial Ave 911, Palo Alto, 94303, California, USA

    Kurt Konolige

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag London

About this chapter

Cite this chapter

Bauer, S., Wasza, J., Lugauer, F., Neumann, D., Hornegger, J. (2013). Real-Time RGB-D Mapping and 3-D Modeling on the GPU Using the Random Ball Cover. In: Fossati, A., Gall, J., Grabner, H., Ren, X., Konolige, K. (eds) Consumer Depth Cameras for Computer Vision. Advances in Computer Vision and Pattern Recognition. Springer, London. https://doi.org/10.1007/978-1-4471-4640-7_2

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-4640-7_2

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-4639-1

  • Online ISBN: 978-1-4471-4640-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation