Abstract
As we become more competent developers of artificially intelligent systems, the level of deployment and associated implicit trust in these systems will increase in kind. While this is an attractive concept, with an already-demonstrated capability to positively disrupt industries around the world, it remains a dangerous premise that demands attention and intentional resource allocation to ensure that these systems’ behaviors match our expectations. Until we can develop explainable AI techniques or high-fidelity simulators to enable us to examine the models’ underlying logic for the situations we intend to utilize them in, it will be irresponsible to place our trust in their ability to act on our behalf. In this work we describe and provide guidelines for ongoing efforts in using novel explainable AI techniques and high-fidelity simulation to help establish shared expectations between autonomous systems and the humans who interact with them, discussing collaborative robotics and cybersecurity domains.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Kypson, A.P., Nifong, L.W., Chitwood Jr., W.R.: Robot-assisted surgery: training and re-training surgeons. Int. J. Med. Robot. Comput. Assist. Surg. 1(1), 70–76 (2004)
Kavak, H., Padilla, J.J., Vernon-Bido, D.: A characterization of cybersecurity simulation scenarios. In: Proceedings of the 19th Communications & Networking Symposium (CNS), p. 3, 3 April 2016
Shah, S., Dey, D., Lovett, C., Kapoor, A.: AirSim: high-fidelity visual and physical simulation for autonomous vehicles. In: Field and Service Robotics 2018, pp. 621–635. Springer, Cham (2018)
Hayes, B., Scassellati, B.: Challenges in shared-environment human-robot collaboration. In: Proceedings of the Collaborative Manipulation Workshop at the ACM/IEEE International Conference on Human-Robot Interaction (HRI), 3 March 2013
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., Mané, D.: Concrete problems in AI safety. arXiv preprint arXiv:1606.06565, 21 June 2016
Nikolaidis, S., Shah, J.: Human-robot cross-training: computational formulation, modeling and evaluation of a human team training strategy. In: 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 33–40. IEEE, 3 March 2013
Fridman, L., Ding, L., Jenik, B., Reimer, B.: Arguing machines: human supervision of black box AI systems that make life-critical decisions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2019)
Confalonieri, R., Besold, T.R., Weyde, T., Creel, K., Lombrozo, T., Mueller, S., Shafto, P.: What makes a good explanation? Cognitive dimensions of explaining intelligent machines. In: CogSci 2019: Creativity + Cognition + Computation (2019)
Tabrez, A., Agrawal, S., Hayes, B.: Explanation-based reward coaching to improve human performance via reinforcement learning. In: 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 249–257. IEEE, 11 March 2019
Doshi-Velez, F., Kim, B.: Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608, 28 February 2017
Besold, T.R., Uckelman, S.L.: The what, the why, and the how of artificial explanations in automated decision-making. arXiv preprint arXiv:1808.07074, 21 August 2018
Mueller, S.T., Hoffman, R.R., Clancey, W., Emrey, A., Klein, G.: Explanation in human-AI systems: a literature meta-review, synopsis of key ideas and publications, and bibliography for explainable AI. arXiv preprint arXiv:1902.01876, 5 February 2019
Hayes, B., Shah, J.A.: Improving robot controller transparency through autonomous policy explanation. In: 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 303–312. IEEE, 6 March 2017
Doran, D., Schulz, S., Besold, T.R.: What does explainable AI really mean? A new conceptualization of perspectives. arXiv preprint arXiv:1710.00794, 2 October 2017
Fonash, P., Schneck, P.: Cybersecurity: from months to milliseconds. Computer 48(1), 42–50 (2015)
Fournaris, A.P., Lalos, A.S., Serpanos, D.: Generative adversarial networks in ai-enabled safety-critical systems: friend or foe? Computer 52(9), 78–81 (2019)
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., Zaremba, W.: Openai gym. arXiv preprint arXiv:1606.01540, 5 June 2016
Matsas, E., Vosniakos, G.C.: Design of a virtual reality training system for human–robot collaboration in manufacturing tasks. Int. J. Interact. Des. Manuf. (IJIDeM) 11(2), 139–153 (2017)
Codd-Downey, R., Forooshani, P.M., Speers, A., Wang, H., Jenkin, M.: From ROS to unity: leveraging robot and virtual environment middleware for immersive teleoperation. In: 2014 IEEE International Conference on Information and Automation (ICIA), pp. 932–936. IEEE, 28 July 2014
Brunner, S.G., Lehner, P., Schuster, M.J., Riedel, S., Belder, R., Leidner, D., Wedler, A., Beetz, M., Stulp, F.: Design, execution, and postmortem analysis of prolonged autonomous robot operations. IEEE Robot. Autom. Lett. 3(2), 1056–1063 (2018)
Schmid, U., Zeller, C., Besold, T., Tamaddoni-Nezhad, A., Muggleton, S.: How does predicate invention affect human comprehensibility? In: The International Conference on Inductive Logic Programming, 4 September 2016, pp. 52–67. Springer, Cham (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Hayes, B., Moniz, M. (2021). Trustworthy Human-Centered Automation Through Explainable AI and High-Fidelity Simulation. In: Cassenti, D., Scataglini, S., Rajulu, S., Wright, J. (eds) Advances in Simulation and Digital Human Modeling. AHFE 2020. Advances in Intelligent Systems and Computing, vol 1206. Springer, Cham. https://doi.org/10.1007/978-3-030-51064-0_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-51064-0_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-51063-3
Online ISBN: 978-3-030-51064-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)