Conceptualizing Overtrust in Robots: Why Do People Trust a Robot That Previously Failed?
In this chapter, we present work that suggests people tend to be overly trusting and overly forgiving of robots in certain situations. In keeping with the theme of this book where intelligent systems help humans recover from errors, our work so far has focused on robots as guides in emergency situations. Our experiments show that, at best, human participants in our simulated emergencies focus on guidance provided by robots, regardless of a robot’s prior performance or other guidance information, and, at worst, believe that the robot is more capable than other sources of information. Even when the robots do break trust, a properly timed statement can convince a participant to follow it. Based on this evidence, we have conceptualized overtrust of robots using our previous framework of situational trust. We define two mechanisms in which people can overtrust robots: misjudging the abilities or intentions of the robot and misjudging the risk in the scenario. We discuss our prior work in light of this new reconceptualization to attempt to explain our previous results and encourage future work.
Support for this research was provided by the Motorola Foundation Professorship, the Linda J. and Mark C. Smith Chair in Bioengineering, Air Force Office of Sponsored Research contract FA9550-13-1-0169 and Georgia Tech Research Institute.
- Bainbridge, W. A., Hart, J. W., Kim, E. S., & Scassellati, B. (2011). The benefits of interactions with physically present robots over video-displayed agents. International Journal of Social Robotics, 3(1), 41-52.Google Scholar
- Desai, M., Kaniarasu, P., Medvedev, M., Steinfeld, A., & Yanco, H. (2013). Impact of robot failures and feedback on real-time trust. Proceedings of the 8th ACM/IEEE international conference on Human-robot interaction, (pp. 251-258). Tokyo, Japan.Google Scholar
- Kim, P. H., Dirks, K. T., Cooper, C. D., & Ferrin, D. L. (2006). When more blame is better than less: The implications of internal vs. external attributions for the repair of trust after a competence-vs. integrity-based trust violation. Organizational Behavior and Human Decision Processes, 99(1), 49-65.Google Scholar
- Robinette, P., Howard, A. M., & Wagner, A. R. (2015). Timing is Key for Robot Trust Repair. International Conference on Social Robotics (pp. 574-583). Paris: Springer International Publishing.Google Scholar
- Robinette, P., Wagner, A. R., & Howard, A. (2014). Assessment of Robot Guidance Modalities Conveying Instructions to Humans in Emergency Situations. Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 14). Edinburgh, UK.Google Scholar
- Robinette, P., Li, W., Allen, R., Howard, A. M., & Wagner, A. R. (2016a). Overtrust of Robots in Emergency Evacuation Scenarios. 11th ACM/IEEE International Conference on Human-Robot Interaction. Christchurch.Google Scholar
- Robinette, P., Wagner, AR., & Howard, AM. (2016b). Assessment of robot to human instruction conveyance modalities across virtual, remote and physical robot presence. Robot and Human Interactive Communication (RO-MAN), 2016 25th IEEE International Symposium on. (pp. 1044-1050). New York City: IEEE.Google Scholar
- Robinette, P., Wagner, A. R., & Howard, A. M. (2016c). Investigating Human-Robot Trust in Emergency Scenarios: Methodological Lessons Learned. In R. Mittu, D. Sofge, A. Wagner, & W. Lawless, Robust Intelligence and Trust in Autonomous Systems (pp. 143-166). Boston: Springer.Google Scholar
- Robinette, P., Wagner, A. R., & Howard, A. M. (2017). The effect of robot performance on human-robot trust in time-critical situations. IEEE Transactions on Human-Machine Systems, PP(99), 1-12.Google Scholar
- Salem, M., Lakatos, G., Amirabdollahian, F., & Dautenhahn, K. (2015). Would You Trust a (Faulty) Robot?: Effects of Error, Task Type and Personality on Human-Robot Cooperation and Trust. Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction, (pp. 141-148). Portland.Google Scholar
- Schweitzer, M. E., Hershey, J. C., & Bradlow, E. T. (2006). Promises and lies: Restoring violated trust. Organizational behavior and human decision processes, 101(1), 1-19.Google Scholar
- Wagner, A. R. (2009). The Role of Trust and Relationships in Human-Robot Social Interaction. Ph.D. diss., School of Interactive Computing, Georgia Institute of Technology, Atlanta, GA.Google Scholar
- Wagner, A. R., & Robinette, P. (2015). Towards Robots that Trust: Human Subject Validation of the Situational Conditions for Trust. Interaction Studies, 16(1), 89-117.Google Scholar