Quantitative Risk Assessment of Safety-Critical Systems via Guided Simulation for Rare Events

  • Stefan PuchEmail author
  • Martin FränzleEmail author
  • Sebastian Gerwinn
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11245)


For developers of assisted or automated driving systems, gaining specific feedback and quantitative figures on the safety impact of the systems under development is crucial. However, obtaining such data from simulation of their design models is a complex and often time-consuming process. Especially when data of interest hinge on extremely rare events, an estimation of potential risks is highly desirable but a non-trivial task lacking easily applicable methods. In this paper we describe how a quantitative statement for a risk estimation involving extremely rare events can be obtained by guiding simulation based on reinforcement learning. The method draws on variance reduction and importance sampling, yet applies different optimization principles than related methods, like the cross-entropy methods against which we compare. Our rationale for optimizing differently is that in quantitative system verification, a sharper upper bound of the confidence interval is of higher relevance than the total width of the confidence interval.

Our application context is deduced from advanced driver assistance system (ADAS) development. In that context virtual driver simulations are performed with the objective to generate quantitative figures for the safety impact in pre-crash situations. In order to clarify the difference of our technique to variance reduction techniques, a comparative evaluation on a simple probabilistic benchmark system is also presented.


  1. 1.
    Clopper, C.J., Pearson, E.S.: The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 26(4), 404–413 (1934)CrossRefGoogle Scholar
  2. 2.
    Donzé, A., Maler, O.: Robust satisfaction of temporal logic over real-valued signals. In: Chatterjee, K., Henzinger, T.A. (eds.) FORMATS 2010. LNCS, vol. 6246, pp. 92–106. Springer, Heidelberg (2010). CrossRefzbMATHGoogle Scholar
  3. 3.
    Efron, B., Tibshirani, R.J.: An Introduction to the Bootstrap. Monographs on Statistics and Applied Probability, no. 57. Chapman & Hall/CRC, London (1993)CrossRefGoogle Scholar
  4. 4.
    European Commission: Towards a European road safety area: policy orientations on road safety 2011–2020 (2010).
  5. 5.
    Eurostat: Slightly over 26 000 victims of road accidents in the EU in 2015. Eurostat Press Office Vincent (2016).
  6. 6.
    Fränzle, M., Hansen, M.R.: A robust interpretation of duration calculus. In: Van Hung, D., Wirsing, M. (eds.) ICTAC 2005. LNCS, vol. 3722, pp. 257–271. Springer, Heidelberg (2005). CrossRefGoogle Scholar
  7. 7.
    Gietelink, O., De Schutter, B., Verhaegen, M.: Adaptive importance sampling for probabilistic validation of advanced driver assistance systems. In: 2006 American Control Conference, vol. 19, 6 pp. (2006)Google Scholar
  8. 8.
    Gietelink, O., De Schutter, B., Verhaegen, M.: Probabilistic validation of advanced driver assistance systems. In: Proceedings of the 16th IFAC World Congress, vol. 19 (2005)CrossRefGoogle Scholar
  9. 9.
    Jegourel, C., Larsen, K.G., Legay, A., Mikučionis, M., Poulsen, D.B., Sedwards, S.: Importance sampling for stochastic timed automata. In: Fränzle, M., Kapur, D., Zhan, N. (eds.) SETTA 2016. LNCS, vol. 9984, pp. 163–178. Springer, Cham (2016). CrossRefGoogle Scholar
  10. 10.
    Kahn, H.: Use of different Monte Carlo sampling techniques, p. 766 (1955)Google Scholar
  11. 11.
    Page, Y., et al.: A comprehensive and harmonized method for assessing the effectiveness of advanced driver assistance systems by virtual simulation: the P.E.A.R.S. initiative. In: The 24th International Technical Conference on the Enhanced Safety of Vehicles (ESV). NHTSA, Gothenburg (2015)Google Scholar
  12. 12.
    Puch, S., Wortelen, B., Fränzle, M., Peikenkamp, T.: Using guided simulation to improve a model-based design process of complex human machine systems. In: Modelling and Simulation, ESM 2012, pp. 159–164. EUROSIS-ETI, Essen (2012)Google Scholar
  13. 13.
    Puch, S., Wortelen, B., Fränzle, M., Peikenkamp, T.: Evaluation of drivers interaction with assistant systems using criticality driven guided simulation. In: Duffy, V.G. (ed.) DHM 2013. LNCS, vol. 8025, pp. 108–117. Springer, Heidelberg (2013). CrossRefGoogle Scholar
  14. 14.
    Rubinstein, R.: The cross-entropy method for combinatorial and continuous optimization. Methodol. Comput. Appl. Probab. 1, 127–190 (1999)Google Scholar
  15. 15.
    Vogel, K.: A comparison of headway and time to collision as safety indicators. Accid. Anal. Prev. 35(3), 427–433 (2003)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Vorndran, I.: Unfallstatistik - Verkehrsmittel im Risikovergleich. DESTATIS (2010).
  17. 17.
    WIVW GmbH: Fahrsimulationssoftware SILAB.
  18. 18.
    Wortelen, B., Baumann, M., Lüdtke, A.: Dynamic simulation and prediction of drivers’ attention distribution. Transp. Res. Part F Traffic Psychol. Behav. 21, 278–294 (2013)CrossRefGoogle Scholar
  19. 19.
    Wortelen, B., Lüdtke, A., Baumann, M.: Integrated simulation of attention distribution and driving behavior. In: Proceedings of the 22nd Annual Conference on Behavior Representation in Modeling & Simulation, pp. 69–76. BRIMS Society, Ottawa (2013)Google Scholar
  20. 20.
    Zuliani, P., Baier, C., Clarke, E.M.: Rare-event verification for stochastic hybrid systems. In: Proceedings of the 15th ACM International Conference on Hybrid Systems: Computation and Control, pp. 217–226. ACM, New York (2012)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Carl von Ossietzky Universität OldenburgOldenburgGermany
  2. 2.OFFIS e.V., Escherweg 2OldenburgGermany

Personalised recommendations