Adaptive Image Processing Methods for Outdoor Autonomous Vehicles

  • Lucie Halodová
  • Eliška Dvořáková
  • Filip Majer
  • Jiří Ulrich
  • Tomáš Vintr
  • Keerthy Kusumam
  • Tomáš KrajníkEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11472)


This paper concerns adaptive image processing for visual teach-and-repeat navigation systems of autonomous vehicles operating outdoors. The robustness and the accuracy of these systems rely on their ability to extract relevant information from the on-board camera images, which is then used for the autonomous navigation and the map building. In this paper, we present methods that allow an image-based navigation system to adapt to a varying appearance of outdoor environments caused by dynamic illumination conditions and naturally occurring environment changes. In the performed experiments, we demonstrate that the adaptive and the learning methods for camera parameter control, image feature extraction and environment map refinement allow autonomous vehicles to operate in real, changing world for extended periods of time.


  1. 1.
    Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (SURF). Comput. Vis. Image Underst. 110(3), 346–359 (2008)CrossRefGoogle Scholar
  2. 2.
    Biber, P., Duckett, T.: Dynamic maps for long-term operation of mobile service robots. In: RSS (2005)Google Scholar
  3. 3.
    Blanc, G., Mezouar, Y., Martinet, P.: Indoor navigation of a wheeled mobile robot along visual routes. In: International Conference on Robotics and Automation (ICRA) (2005)Google Scholar
  4. 4.
    Calonder, M., Lepetit, V., Strecha, C., Fua, P.: BRIEF: binary robust independent elementary features. In: Proceedings of the ICCV (2010)Google Scholar
  5. 5.
    Carlevaris-Bianco, N., Eustice, R.M.: Learning visual feature descriptors for dynamic lighting conditions. In: IROS. IEEE (2014)Google Scholar
  6. 6.
    Chen, Z., Birchfield, S.T.: Qualitative vision-based path following. IEEE Trans. Rob. Autom. 25(3), 749–754 (2009)CrossRefGoogle Scholar
  7. 7.
    Churchill, W.S., Newman, P.: Experience-based navigation for long-term localisation. IJRR 32(14), 1645–1661 (2013). Scholar
  8. 8.
    Dayoub, F., Duckett, T.: An adaptive appearance-based map for long-term topological localization of mobile robots. In: IROS (2008)Google Scholar
  9. 9.
    De Cristóforis, P., et al.: Real-time monocular image-based path detection. J. Real Time Image Process. 11, 335–348 (2013)CrossRefGoogle Scholar
  10. 10.
    Debevec, P.E., Malik, J.: Recovering high dynamic range radiance maps from photographs. In: SIGGRAPH. ACM (2008)Google Scholar
  11. 11.
    DeSouza, G.N., Kak, A.C.: Vision for mobile robot navigation: A survey. IEEE Trans. Pattern Anal. Mach. Intell. 24(2), 237–267 (2002). Scholar
  12. 12.
    Dvořáková, E.: Temporal models for mobile robot visual navigation. B.S. thesis, Czech Technical Univerzity in Prague (2018)Google Scholar
  13. 13.
    Engel, J., Schöps, T., Cremers, D.: LSD-SLAM: large-scale direct monocular SLAM. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8690, pp. 834–849. Springer, Cham (2014). Scholar
  14. 14.
    Gadd, M., Newman, P.: Checkout my map: Version control for fleetwide visual localisation. In: IROS. IEEE/RSJ (2016)Google Scholar
  15. 15.
    Halodová, L.: Map management for long-term navigation of mobile robots. Bachelor thesis, Czech Technical University, May 2018Google Scholar
  16. 16.
    Halodová, L., Krajník, T.: Exposure setting for visual navigation of mobile robots. In: Student Conference on Planning in AI and Robotics (PAIR) (2017)Google Scholar
  17. 17.
    Holmes, S., Klein, G., Murray, D.W.: A square root unscented Kalman filter for visual monoSLAM. In: International Conference on Robotics and Automation (ICRA) (2008)Google Scholar
  18. 18.
    Kosaka, A., Kak, A.C.: Fast vision-guided mobile robot navigation using model-based reasoning and prediction of uncertainties. CVGIP: Image Underst. 56(3), 271–329 (1992)CrossRefGoogle Scholar
  19. 19.
    Krajník, T., Cristóforis, P., Nitsche, M., Kusumam, K., Duckett, T.: Image features and seasons revisited. In: European Conference on Mobile Robots (ECMR) (2015)Google Scholar
  20. 20.
    Krajník, T., Majer, F., Halodová, L., Vintr, T.: Navigation without localisation: reliable teach and repeat based on the convergence theorem. In: IROS (2018)Google Scholar
  21. 21.
    Krajník, T., et al.: Image features for visual teach-and-repeat navigation in changing environments. Rob. Auton. Syst. 88, 127–141 (2017)CrossRefGoogle Scholar
  22. 22.
    Krajník, T., et al.: Long-term topological localization for service robots in dynamic environments using spectral maps. In: IROS (2014)Google Scholar
  23. 23.
    Krajník, T., et al.: FreMEN: frequency map enhancement for long-term mobile robot autonomy in changing environments. IEEE Trans. Rob. 33(4), 964–977 (2017)CrossRefGoogle Scholar
  24. 24.
    Kunze, L., Hawes, N., Duckett, T., Hanheide, M., Krajnik, T.: Artificial intelligence for long-term robot autonomy: a survey. IEEE RAL 3(4), 4023–4030 (2018). Scholar
  25. 25.
    Latif, Y., Garg, R., Milford, M., Reid, I.: Addressing challenging place recognition tasks using generative adversarial networks. In: ICRA (2018)Google Scholar
  26. 26.
    Linegar, C., Churchill, W., Newman, P.: Work smart, not hard: recalling relevant experiences for vast-scale but time-constrained localisation. In: ICRA (2015)Google Scholar
  27. 27.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  28. 28.
    Lowry, S., Milford, M.J.: Supervised and unsupervised linear learning techniques for visual place recognition in changing environments. IEEE T-RO 32(3), 600–613 (2016)CrossRefGoogle Scholar
  29. 29.
    Lowry, S., et al.: Visual place recognition: a survey. IEEE T-RO 32(1), 1–19 (2016)CrossRefGoogle Scholar
  30. 30.
    Lu, H., Zhang, H., Yang, S., Zheng, Z.: Camera parameters auto-adjusting technique for robust robot vision. In: ICRA. IEEE (2010)Google Scholar
  31. 31.
    Mair, E., Hager, G.D., Burschka, D., Suppa, M., Hirzinger, G.: Adaptive and generic corner detection based on the accelerated segment test. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6312, pp. 183–196. Springer, Heidelberg (2010). Scholar
  32. 32.
    Majer, F., Halodová, L., Krajník, T.: Source codes: bearing-only navigation.
  33. 33.
    Majer, F., et al.: A versatile visual navigation system for outdoor autonomous vehicles. In: Modeling and Simulation for Autonomous Systems (2018, in review)Google Scholar
  34. 34.
    Matsumoto, Y., Inaba, M., Inoue, H.: Visual navigation using view-sequenced route representation. In: International Conference on Robotics and Automation (ICRA) (1996)Google Scholar
  35. 35.
    Mühlfellner, P., Bürki, M., Bosse, M., Derendarz, W., Philippsen, R., Furgale, P.: Summary maps for lifelong visual localization. J. Field Rob. 33(5), 561–590 (2016)CrossRefGoogle Scholar
  36. 36.
    Mur-Artal, R., Montiel, J.M.M., Tardós, J.D.: ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE Trans. Rob. 31(5), 1147–1163 (2015). Scholar
  37. 37.
    Neubert, P., Sunderhauf, N., Protzel, P.: Appearance change prediction for long-term navigation across seasons. In: European Conference on Mobile Robotics (2013)Google Scholar
  38. 38.
    Neves, A.J.R., Cunha, B., Pinho, A.J., Pinheiro, I.: Autonomous configuration of parameters in robotic digital cameras. In: Araujo, H., Mendonça, A.M., Pinho, A.J., Torres, M.I. (eds.) IbPRIA 2009. LNCS, vol. 5524, pp. 80–87. Springer, Heidelberg (2009). Scholar
  39. 39.
    Paton, M., MacTavish, K., Berczi, L.-P., van Es, S.K., Barfoot, T.D.: I can see for miles and miles: an extended field test of visual teach and repeat 2.0. In: Hutter, M., Siegwart, R. (eds.) Field and Service Robotics. SPAR, vol. 5, pp. 415–431. Springer, Cham (2018). Scholar
  40. 40.
    Porav, H., Maddern, W., Newman, P.: Adversarial training for adverse conditions: robust metric localisation using appearance transfer. In: ICRA (2018)Google Scholar
  41. 41.
    Rosen, D.M., Mason, J., Leonard, J.J.: Towards lifelong feature-based mapping in semi-static environments. In: ICRA. IEEE (2016)Google Scholar
  42. 42.
    Royer, E., Lhuillier, M., Dhome, M., Lavest, J.M.: Monocular vision for mobile robot localization and autonomous navigation. Int. J. Comput. Vis. 74(3), 237–260 (2007)CrossRefGoogle Scholar
  43. 43.
    Segvic, S., Remazeilles, A., Diosi, A., Chaumette, F.: Large scale vision based navigation without an accurate global reconstruction. In: CVPR (2007)Google Scholar
  44. 44.
    Shim, I., Lee, J.Y., Kweon, I.S.: Auto-adjusting camera exposure for outdoor robotics using gradient information. In: IROS. IEEE/RSJ (2014)Google Scholar
  45. 45.
    Krajník, T., Faigl, J., Vonásek, V., et al.: Simple, yet stable bearing-only navigation. J. Field Rob. 27(5), 511–533 (2010)CrossRefGoogle Scholar
  46. 46.
    Zhang, N., Warren, M., Barfoot, T.: Learning place-and-time-dependent binary descriptors for long-term visual localization. In: ICRA. IEEE (2016)Google Scholar
  47. 47.
    Zhang, Z., Forster, C., Scaramuzza, D.: Active exposure control for robust visual odometry in HDR environments. In: ICRA (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Artificial Intelligence Center, FEECzech Technical UniversityPragueCzech Republic
  2. 2.University of NottinghamNottinghamUK

Personalised recommendations