Structuring Validation Targets of a Machine Learning Function Applied to Automated Driving

  • Lydia GauerhofEmail author
  • Peter Munk
  • Simon Burton
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11093)


The validation of highly automated driving vehicles is an important challenge to the automotive industry, since even if the system is free from internal faults, its behaviour might still vary from the original intent. Reasons for these deviations from the intended functionality can be found in the unpredictability of environmental conditions as well the intrinsic uncertainties of the Machine Learning (ML) functions used to make sense of this complex input space.

In this paper, we propose a safety assurance case for a pedestrian detection function, a safety-relevant baseline functionality for an automated driving system. Our safety assurance case is presented in the graphical structuring notation (GSN) and combines our arguments against the problems of underspecification [9], the semantic gap [3], and the deductive gap [16].


Safety Intended functionality Functional insufficiency Nominal performance Automated driving Machine learning Assurance case GSN 


  1. 1.
    Alsallakhl, B., Jourabloo, A., Ye, M., Liu, X., Ren, L.: Do convolutional neural networks learn class hierarchy? IEEE Trans. Vis. Comput. Graph. 24(1), 152–162 (2018). Scholar
  2. 2.
    Amarnath, R., Munk, P., Thaden, E., Nordmann, A., Burton, S.: Dependability challenges in the model-driven engineering of automotive systems. In: 2016 IEEE International Symposium on Software Reliability Engineering Workshops, ISSREW, pp. 1–4, October 2016Google Scholar
  3. 3.
    Bergenhem, C., et al.: How to reach complete safety requirement refinement for autonomous vehicles. Technical report, CARS 2015 - Critical Automotive Applications: Robustness & Safety, Paris, France, September 2015Google Scholar
  4. 4.
    Burton, S., Gauerhof, L., Heinzemann, C.: Making the case for safety of machine learning in highly automated driving. In: Tonetta, S., Schoitsch, E., Bitsch, F. (eds.) SAFECOMP 2017. LNCS, vol. 10489, pp. 5–16. Springer, Cham (2017). Scholar
  5. 5.
    Evtimov, I., et al.: Robust physical-world attacks on deep learning models. Cryptography and Security (2017).
  6. 6.
    Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples (2015).
  7. 7.
    Katz, G., Barrett, C., Dill, D., Julian, K., Kochenderfer, M.: Reluplex: an efficient SMT solver for verifying deep neural networks. Technical report. Stanford University, USA (2017).
  8. 8.
    Kendall, A., Gal, Y.: What uncertainties do we need in Bayesian deep learning for computer vision? (2017).
  9. 9.
    Koopman, P., Wagner, M.: Challenges in autonomous vehicle testing and validation. SAE Int. J. Trans. Saf. 4, 15–24 (2016). Scholar
  10. 10.
    Krizhevsky, A., Sutskever, I., Hinton., G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  11. 11.
  12. 12.
    Leveson, N.G.: Engineering a Safer World: Systems Thinking Applied to Safety. The MIT Press, Cambridge (2011)Google Scholar
  13. 13.
    Nguyen, A.M., Yosinski, J., Clune, J.: Multifaceted feature visualization: uncovering the different types of features learned by each neuron in deep neural networks. CoRR abs/1602.03616 (2016).
  14. 14.
    Tas, Ö.Ş., Kuhnt, F., Zllner, J.M., Stiller, C.: Functional system architectures towards fully automated driving. In: 2016 IEEE Intelligent Vehicles Symposium, IV (2016).
  15. 15.
    Samuel, A.L.: Some studies in machine learning using the game of checkers. IBM J. Res. Dev. 3(3), 210–229 (1959). Scholar
  16. 16.
    Wilhelm, U., Ebel, S., Weitzel, A.: Functional safety of driver assistance systems and ISO 26262. In: Winner, H., Hakuli, S., Lotz, F., Singer, C. (eds.) Handbook of Driver Assistance Systems, pp. 109–131. Springer, Cham (2016). Scholar
  17. 17.
    Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Corporate Research, Robert Bosch GmbHRenningenGermany

Personalised recommendations