An Initial Generic Assessment Framework for the Consideration of Risk in the Implementation of Autonomous Systems
This paper considers some of the issues around autonomous systems and the different types of risk involved in their implementation. These risks are both barriers to the implementation of a successful autonomous system and risks that are consequences of the use of such systems. The different levels of automation, and different approaches to categorizing these levels, as presented in a variety of frameworks, are summarized and discussed.
The paper presents an initial generic assessment structure, with the aim of providing a useful construct for the design and development of acceptable autonomous systems that are intended to replace elements of the human cognitive process, specifically in situations involving decision-making. It introduces the concept of the “logos chasm”: the gap between achievable autonomous systems and those which currently only exist in the realm of science fiction; and discusses possible reasons for its existence.
KeywordsAutonomous systems Automation risks Automation frameworks
- 2.Sheridan, T.B.: Telerobotics, Automation, and Human Supervisory Control. MIT Press, Cambridge (1992)Google Scholar
- 4.Proud, R.W., Hart, J.J.: FLOAAT, A Tool for Determining Levels of Autonomy and Automation, Applied to Human-Rated Space Systems. Infotech@Aerospace, American Institute of Aeronautics and Astronautics (AIAA) 2005-7061. AIAA, Arlington (2005)Google Scholar
- 5.Osinga, F.: Science, Strategy and War: The Strategic Theory of John Boyd. Eburon Academic Publishers, Delft, The Netherlands (2005)Google Scholar
- 8.Cramer, H.S., Evers, V., Van Someren, M.W., Wielinga, B.J.: Awareness, training and trust in interaction with adaptive spam filters. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 909–912. ACM (2009)Google Scholar
- 10.Cresci, E.: Chatbot that overturned 160,000 parking fines now helping refugees claim asylum. The Guardian (2017). https://www.theguardian.com/technology/2017/mar/06/chatbot-donotpay-refugees-claim-asylum-legal-aid
- 12.Caringi, A.: The FDA approved the world’s first autonomous insulin pump vocativ: health (2016). http://www.vocativ.com/363955/the-fda-approved-the-worlds-first-autonomous-insulin-pump/index.html
- 13.Platania, R., Shams, S., Yang, S., Zhang, J., Lee, K., Park, S-J.: Automated breast cancer diagnosis using deep learning and region of interest detection (BC-DROID). In: Proceedings of the 8th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB 2017), pp. 536–543. ACM, New York (2017). https://doi.org/10.1145/3107411.3107484
- 14.J3016: Taxonomy and Definitions for Terms Related to On-Road Motor Vehicle Automated Driving Systems. SAE International (2014)Google Scholar
- 15.Bloom, B.S., Engelhart, M.D., Furst, E.J., Hill, W.H., Krathwohl, D.R.: Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. David McKay Company, New York (1956)Google Scholar
- 17.Shelburne, W.A.: Mythos and Logos in the Thought of Carl Jung: The theory of the Collective Unconscious in Scientific Perspective. State University of New York Press, Albany N. Y (1988)Google Scholar