Uncertainty in Machine Learning: A Safety Perspective on Autonomous Driving

  • Sina ShafaeiEmail author
  • Stefan Kugele
  • Mohd Hafeez Osman
  • Alois Knoll
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11094)


With recent efforts to make vehicles intelligent, solutions based on machine learning have been accepted to the ecosystem. These systems in the automotive domain are growing fast, speeding up the promising future of highly and fully automated driving, and respectively, raising new challenges regarding safety assurance approaches. Uncertainty in data and the machine learning methods is a key point to investigate one of the main origins of safety-related concerns. In this work, we inspect this issue in the domain of autonomous driving with an emphasis on four safety-related cases, then introduce our proposals to address the challenges and mitigate them. The core of our approach is on introducing monitoring limiters during development time of such intelligent systems.


Artificial intelligence Uncertainty Safety 


  1. 1.
    Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., Mané, D.: Concrete problems in AI safety. CoRR abs/1606.06565 (2016)Google Scholar
  2. 2.
    Blei, D.M., Kucukelbir, A., McAuliffe, J.D.: Variational inference: a review for statisticians. J. Am. Stat. Assoc. 112(518), 859–877 (2017). Scholar
  3. 3.
    Burton, S., Gauerhof, L., Heinzemann, C.: Making the case for safety of machine learning in highly automated driving. In: Tonetta, S., Schoitsch, E., Bitsch, F. (eds.) SAFECOMP 2017. LNCS, vol. 10489, pp. 5–16. Springer, Cham (2017). Scholar
  4. 4.
    Feld, M., Müller, C.: The automotive ontology: managing knowledge inside the vehicle and sharing it between cars. In: Proceedings of the 3rd International Conference on Automotive User Interfaces and Interactive Vehicular Applications, pp. 79–86. ACM (2011).
  5. 5.
    Huang, X., Kwiatkowska, M., Wang, S., Wu, M.: Safety verification of deep neural networks. In: Majumdar, R., Kunčak, V. (eds.) CAV 2017. LNCS, vol. 10426, pp. 3–29. Springer, Cham (2017). Scholar
  6. 6.
    International Organization for Standardization: ISO 26262: Road vehicles-functional safety. International Standard ISO/FDIS 26262 (2011)Google Scholar
  7. 7.
    Kane, A., Chowdhury, O., Datta, A., Koopman, P.: A case study on runtime monitoring of an autonomous research vehicle (ARV) system. In: Bartocci, E., Majumdar, R. (eds.) RV 2015. LNCS, vol. 9333, pp. 102–117. Springer, Cham (2015). Scholar
  8. 8.
    McAllister, R., et al.: Concrete problems for autonomous vehicle safety: advantages of Bayesian deep learning. In: International Joint Conferences on Artificial Intelligence, Inc. (2017).
  9. 9.
    Pei, K., Cao, Y., Yang, J., Jana, S.: Towards practical verification of machine learning: The case of computer vision systems. CoRR abs/1712.01785 (2017)Google Scholar
  10. 10.
    Russell, S., Dewey, D., Tegmark, M.: Research priorities for robust and beneficial artificial intelligence. AI Mag. 36(4), 105–114 (2015)CrossRefGoogle Scholar
  11. 11.
    Salay, R., Queiroz, R., Czarnecki, K.: An analysis of ISO 26262: Using machine learning safely in automotive software. CoRR abs/1709.02435 (2017)Google Scholar
  12. 12.
    Sölch, M., Bayer, J., Ludersdorfer, M., van der Smagt, P.: Variational inference for on-line anomaly detection in high-dimensional time series. CoRR abs/1602.07109 (2016)Google Scholar
  13. 13.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. Adaptive Computation and Machine Learning. MIT Press, Cambridge (1998)Google Scholar
  14. 14.
    Taylor, J., Yudkowsky, E., LaVictoire, P., Critch, A.: Alignment for advanced machine learning systems. Machine Intelligence Research Institute (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Technical University of MunichMunichGermany

Personalised recommendations