Skip to main content
SpringerLink
Log in

Into Darkness: Visual Navigation Based on a Lidar-Intensity-Image Pipeline

  • Chapter
  • First Online:
Robotics Research

Part of the book series: Springer Tracts in Advanced Robotics ((STAR,volume 114))

Abstract

Visual navigation of mobile robots has become a core capability that enables many interesting applications from planetary exploration to self-driving cars. While systems built on passive cameras have been shown to be robust in well-lit scenes, they cannot handle the range of conditions associated with a full diurnal cycle. Lidar, which is fairly invariant to ambient lighting conditions, offers one possible remedy to this problem. In this paper, we describe a visual navigation pipeline that exploits lidar’s ability to measure both range and intensity (a.k.a., reflectance) information. In particular, we use lidar intensity images (from a scanning-laser rangefinder) to carry out tasks such as visual odometry (VO) and visual teach and repeat (VT&R) in realtime, from full-light to full-dark conditions. This lighting invariance comes at the price of coping with motion distortion, owing to the scanning-while-moving nature of laser-based imagers. We present our results and lessons learned from the last few years of research in this area.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abymar, T., Hartl, F., Hirzinger, G., Burschka, D., Frohlich, C.: Automatic registration of panoramic 2.5D scans and color images. In: Proceedings of the International Calibration and Orientation Workshop EuroCOW, vol. 54, Castelldefels, Spain (2007)

    Google Scholar 

  2. Anderson, S., Barfoot, T.D.: RANSAC for motion-distorted 3D visual sensors. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Tokyo, Japan (2013)

    Google Scholar 

  3. Anderson, S., Barfoot, T.D.: Towards relative continuous-time SLAM. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 1025–1032, Karlsruhe, Germany (2013)

    Google Scholar 

  4. Anderson, S., McManus, C., Dong, H., Beerepoot, E., Barfoot, T.D.: The gravel pit lidar-intensity imagery datase. Technical report ASRL-2012-ABL001, University of Toronto (2012)

    Google Scholar 

  5. Bay, H., Ess, A., Tuytelaars, T., Gool, L.: SURF: speeded up robust features. Comput. Vis. Image Underst. (CVIU) 110(3), 346–359 (2008)

    Article  Google Scholar 

  6. Bohm, J., Becker, S.: Automatic marker-free registration of terrestrial laser scans using reflectance features. In: Proceedings of the 8th Conference on Optical 3D Measurement Techniques, pp. 338–344, Zurich, Switzerland (2007)

    Google Scholar 

  7. Brown, D.C.: A solution to the general problem of multiple station analytical stereotriangulation. Rca-mtp data reduction technical report no. 43, Patrick Airforce Base, Florida (1958)

    Google Scholar 

  8. Dold, C., Brenner, C.: Registration of terrestrial laser scanning data using planar patches and image data. In: Proceedings of the ISPRS Commission V Symposium ’Image Engineering and Vision Metrology’, vol. XXXVI, Dresden, Germany (2006)

    Google Scholar 

  9. Dong, H., Anderson, S., Barfoot, T.D.: Two-axis scanning lidar geometric calibration using intensity imagery and distortion mapping. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 3657–3663, Karlsruhe, Germany (2013)

    Google Scholar 

  10. Dong, H.J., Barfoot, T.D.: Lighting-invariant visual odometry using lidar intensity imagery and pose interpolation. In: Proceedings of the International Conference on Field and Service Robotics (FSR), Matsushima, Japan (2012)

    Google Scholar 

  11. Droeschel, D., Holz, D., Stuckler, J., Behnke, S.: Using time-of-flight cameras with active gaze control for 3D collision avoidance. In: Proceedings of the IEEE International Conference on Robotics and Automation, Anchorage, Alaska, United States (2010)

    Google Scholar 

  12. Fischler, M., Bolles, R.: Random sample and consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  13. Furgale, P., Rehder, J., Siegwart, R.: Unified temporal and spatial calibration for multi-sensor systems. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Tokyo, Japan (2013)

    Google Scholar 

  14. Furgale, P.T., Barfoot, T.D.: Visual teach and repeat for long-range rover autonomy. J. Field Robot., Special issue on visual mapping and navigation outdoors, 27(5):534–560 (2010)(video1), (video2), (video3)

    Google Scholar 

  15. Furgale, P.T., Barfoot, T.D., Sibley, G.: Continuous-time batch estimation using temporal basis functions. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 2088–2095, St. Paul, USA (2012)

    Google Scholar 

  16. Gammell, J.D., Tong, C.H., Barfoot, T.D.: Blinded by the light: exploiting the deficiencies of a laser rangefinder for rover attitude estimation. In: Proceedings of the 10th Conference on Computer and Robot Vision (CRV), pp. 144–150, Regina, Canada (2013)

    Google Scholar 

  17. Gammell, J.D., Tong, C.H., Berczi, P., Anderson, S., Barfoot, T.D., Enright, J.: Rover odometry aided by a star tracker. In: Proceedings of the IEEE Aerospace Conference, pp. 1–10, Big Sky, MT (2013)

    Google Scholar 

  18. Kretschmer, U., Abymar, T., Thies, M., Frohlich, C.: Traffic construction analysis by use of terrestrial laser scanning. In: Proceedings of the ISPRS WG VIII/2 Laser-Scanners for Forest and Landscape Assessment, vol. XXXVI, pp. 232–236, Freiburg, Germany (2004)

    Google Scholar 

  19. Lambert, A., Furgale, P.T., Barfoot, T.D., Enright, J.: Field testing of visual odometry aided by a sun sensor and inclinometer. J. Field Robot., Special issue on Space Robotics, 29(3):426–444 (2012) (video)

    Google Scholar 

  20. May, S., Fuchs, S., Malis, E., Nuchter, A., Hertzberg, J.: Three-dimensional mapping with time-of-flight cameras. J. Field Robot. 26(11–12), 934–965 (2009)

    Article  Google Scholar 

  21. McManus, C., Furgale, P.T., Barfoot, T.D.: Towards appearance-based methods for lidar sensors. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 1930–1935, Shanghai, China (2011)

    Google Scholar 

  22. McManus, C., Furgale, P.T., Stenning, B.E., Barfoot, T.D.: Visual teach and repeat using appearance-based lidar. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 389–396, St. Paul, USA (2012)

    Google Scholar 

  23. McManus, C., Furgale, P.T., Barfoot, T.D.: Towards lighting-invariant visual navigation: an appearance-based approach using scanning laser-rangefinders. Robot. Auton. Syst. 61, 836–852 (2013)

    Google Scholar 

  24. McManus, C., Furgale, P.T., Stenning, B.E., Barfoot, T.D.: Lighting-invariant visual teach and repeat using appearance-based lidar. J. Field Robot. 30(2), 254–287 (2013)

    Google Scholar 

  25. Oth, L., Furgale, P., Kneip, L., Siegwart, R.: Rolling shutter camera calibration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1360–1367 (2013)

    Google Scholar 

  26. Tong, C., Dong, H., Barfoot, T.D.: Pose interpolation for laser-based visual odometry. Submitted to the J. Field Robot., 27 May 2013. Manuscript # ROB-13-0044

    Google Scholar 

  27. Tong, C.H., Barfoot, T.D.: Gaussian process Gauss-Newton for 3D laser-based visual odometry. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 5184–5191, Karlsruhe, Germany (2013)

    Google Scholar 

  28. Tong, C.H., Furgale, P.T., Barfoot, T.D.: Gaussian process Gauss-Newton: non-parametric state estimation. In: Proceedings of the 9th Conference on Computer and Robot Vision (CRV), pp. 206–213, Toronto, Canada (2012)

    Google Scholar 

  29. Tong, C.H., Furgale, P.T., Barfoot, T.D.: Gaussian process Gauss-Newton for non-parametric simultaneous localization and mapping. Int. J. Robot. Res. 32(5), 507–525 (2013)

    Google Scholar 

  30. Weingarten, J.W., Gruner, G., Siegwart, R.: A state-of-the-art 3D sensor for robot navigation. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robotics and Systems, vol. 3, pp. 2155–2160, Lasuanne, Switzerland (2004)

    Google Scholar 

  31. Ye, C., Bruch, M.: A visual odometry method based on the SwissRanger SR4000. In: Proceedings of the SPIE—Unmanned Systems Technology XII, vol. 7692 (2010)

    Google Scholar 

  32. Yuan, F., Swadzba, A., Philippsen, R., Engin, O., Hanheide, M., Wachsmuth, S.: Laser-based navigation enhanced with 3D time-of-flight data. In: Proceedings of the IEEE International Conference on Robotics and Automation, Kobe, Japan (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Toronto Institute for Aerospace Studies, 4925 Dufferin St., Toronto, ON, Canada

    Timothy D. Barfoot, Colin McManus, Sean Anderson, Hang Dong, Erik Beerepoot, Chi Hay Tong, Paul Furgale, Jonathan D. Gammell & John Enright

Authors
  1. Timothy D. Barfoot
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Colin McManus
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sean Anderson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hang Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Erik Beerepoot
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chi Hay Tong
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Paul Furgale
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jonathan D. Gammell
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. John Enright
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Timothy D. Barfoot .

Editor information

Editors and Affiliations

  1. Creative Informatics, The University of Tokyo, Tokyo, Japan

    Masayuki Inaba

  2. School of Electrical Engineering and Com, Queensland Univ of Technology, Brisbane, Queensland, Australia

    Peter Corke

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Barfoot, T.D. et al. (2016). Into Darkness: Visual Navigation Based on a Lidar-Intensity-Image Pipeline. In: Inaba, M., Corke, P. (eds) Robotics Research. Springer Tracts in Advanced Robotics, vol 114. Springer, Cham. https://doi.org/10.1007/978-3-319-28872-7_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-28872-7_28

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-28870-3

  • Online ISBN: 978-3-319-28872-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation