Concerns on the Differences Between AI and System Safety Mindsets Impacting Autonomous Vehicles Safety

  • A. M. NascimentoEmail author
  • L. F. Vismari
  • P. S. Cugnasca
  • J. B. CamargoJr.
  • J. R. de AlmeidaJr.
  • R. Inam
  • E. Fersman
  • A. Hata
  • M. V. Marquezini
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11094)


The inflection point in the development of some core technologies enabled the Autonomous Vehicles (AV). The unprecedented growth rate in Artificial Intelligence (AI) and Machine Learning (ML) capabilities, focusing only on AVs, is expected to shift the transportation paradigm and bring relevant benefits to the society, such as accidents reduction. However, recent AVs accidents resulted in life losses. This paper presents a viewpoint discussion based on findings from a preliminary exploratory literature review. It was identified an important misalignment between AI and Safety research communities regarding the impact of AI on the safety risks in AV. This paper promotes this discussion, raises concerns on the potential consequences and suggests research topics to reduce the differences between AI and system safety mindsets.


Autonomous vehicles Safety Artificial intelligence Accident Risks Misalignment Mindset 



This work is supported by the Research, Development and Innovation Center, Ericsson Telecomunicações S.A., Brazil.


  1. 1.
    Singh, S.: Critical reasons for crashes investigated in the National Motor Vehicle Crash Causation Survey, Washington, DC (2015)Google Scholar
  2. 2.
    Dogan, E., Chatila, R., Chauvier, S., Evans, K., Hadjixenophontos, P., Perrin, J.: Ethics in the design of automated vehicles: the AVEthics project. In: EDIA@ ECAI, pp. 10–13 (2016)Google Scholar
  3. 3.
    Meltz, D., Guterman, H.: RobIL—Israeli program for research and development of autonomous UGV: performance evaluation methodology. In: IEEE International Conference on the Science of Electrical Engineering (ICSEE), pp. 1–5 (2016)Google Scholar
  4. 4.
    Lugano, G.: Virtual assistants and self-driving cars. In: 2017 15th International Conference on ITS Telecommunications (ITST), pp. 1–5 (2017)Google Scholar
  5. 5.
    Hamdi, S., Faiedh, H., Souani, C., Besbes, K.: Road signs classification by ANN for real-time implementation. In: 2017 International Conference on Control, Automation and Diagnosis (ICCAD), pp. 328–332 (2017)Google Scholar
  6. 6.
    Yoneda, K., Kuramoto, A., Suganuma, N.: Convolutional neural network based vehicle turn signal recognition. In: 2017 International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS), pp. 204–205 (2017)Google Scholar
  7. 7.
    Chen, Z., Huang, X.: Accurate and reliable detection of traffic lights using multiclass learning and multiobject tracking. IEEE Intell. Transp. Syst. Mag. 8, 28–42 (2016)CrossRefGoogle Scholar
  8. 8.
    Habermann, D., Vido, C.E.O., Osório, F.S., Ramos, F.: Road junction detection from 3D point clouds. In: 2016 International Joint Conference on Neural Networks (IJCNN), pp. 4934–4940 (2016)Google Scholar
  9. 9.
    Hossai, M.R.T., Shahjalal, M.A., Nuri, N.F.: Design of an IoT based autonomous vehicle with the aid of computer vision. In: International Conference on Electrical, Computer and Communication Engineering (ECCE), pp. 752–756 (2017)Google Scholar
  10. 10.
    Bock, J., Beemelmanns, T., Klösges, M., Kotte, J.: Self-learning Trajectory Prediction with Recurrent Neural Networks at Intelligent Intersections (2017)Google Scholar
  11. 11.
    Lopez Pulgarin, E.J., Herrmann, G., Leonards, U.: Drivers’ Manoeuvre classification for safe HRI. In: Gao, Y., Fallah, S., Jin, Y., Lekakou, C. (eds.) TAROS 2017. LNCS (LNAI), vol. 10454, pp. 475–483. Springer, Cham (2017). Scholar
  12. 12.
    Koopman, P.: Challenges in autonomous vehicle validation: keynote presentation abstract. In: Proceedings of the 1st International Workshop on Safe Control of Connected and Autonomous Vehicles, p. 3 (2017)Google Scholar
  13. 13.
    McAllister, R., et al.: Concrete problems for autonomous vehicle safety: advantages of Bayesian deep learning. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence, IJCAI 2017 (2017)Google Scholar
  14. 14.
    NTSB: Collision Between a Car Operating with Automated Vehicle Control Systems and a Tractor-Semitrailer Truck Near Williston, Florida 7 May 2016.
  15. 15.
  16. 16.
    NTSB: Driver Errors, Overreliance on Automation, Lack of Safeguards, Led to Fatal Tesla Crash.
  17. 17.
    Leveson, N.G.: The use of safety cases in certification and regulation (2011)Google Scholar
  18. 18.
  19. 19.
    Mirnig, N., Stollnberger, G., Miksch, M., Stadler, S., Giuliani, M., Tscheligi, M.: To err is robot: how humans assess and act toward an erroneous social robot. Front. Robot. AI. 4, 21 (2017)CrossRefGoogle Scholar
  20. 20.
    Clarke, R.: Big data, big risks. Inf. Syst. J. 26, 77–90 (2016)CrossRefGoogle Scholar
  21. 21.
    Liu, K., Sun, L., Dix, A., Narasipuram, M.: Norm-based agency for designing collaborative information systems. Inf. Syst. J. 11, 229–247 (2001)CrossRefGoogle Scholar
  22. 22.
    Committee on Legal Affairs: REPORT with recommendations to the Commission on Civil Law Rules on Robotics (2017)Google Scholar
  23. 23.
    Delcker, J.: Europe divided over robot “personhood,” (2018)Google Scholar
  24. 24.
    King, W.R.: Editor’s comment decision support systems, artificial intelligence, and expert systems. MIS Q. 8, iv–v (1984)Google Scholar
  25. 25.
  26. 26.
    Harris, M.: Google reports self-driving car mistakes: 272 failures and 13 near misses (2016)Google Scholar
  27. 27.
    Wakabayashi, D.: Self-Driving Uber Car Kills Pedestrian in Arizona, Where Robots Roam (2018)Google Scholar
  28. 28.
    BBC: Tesla in fatal California crash was on Autopilot (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • A. M. Nascimento
    • 1
    Email author
  • L. F. Vismari
    • 1
  • P. S. Cugnasca
    • 1
  • J. B. CamargoJr.
    • 1
  • J. R. de AlmeidaJr.
    • 1
  • R. Inam
    • 2
  • E. Fersman
    • 2
  • A. Hata
    • 3
  • M. V. Marquezini
    • 3
  1. 1.School of EngineeringUniversity of São Paulo (USP)São PauloBrazil
  2. 2.Ericsson ResearchEricsson ABStockholmSweden
  3. 3.Ericsson Telecomunicações S.A.IndaiatubaBrazil

Personalised recommendations