Skip to main content
SpringerLink
Log in

Research Considerations and Tools for Evaluating Human-Automation Interaction with Future Unmanned Systems

  • Chapter
  • First Online:
Autonomy and Artificial Intelligence: A Threat or Savior?

Abstract

Advances in automation will soon enable a single operator to supervise multiple unmanned aerial vehicles. Successfully implementing this new supervisory control paradigm not only requires improvements in automation capability and reliability, but also an understanding of the human performance issues associated with concurrent management of several automated systems. Research in this area has generally focused on topics such as trust, reliability, and levels of automation. The goal of automating systems is generally to minimize the human’s need to directly interact with the system; despite this objective, the majority of current supervisory control research emphasizes situations in which the human must frequently interact with the automation. This is typically done to provide researchers with a clear means of assessing human performance, but ultimately limits the generalizability of the research since it only applies to a limited mission context. The current chapter discusses a model of assessing human-automation interaction that emphasizes not only the traditional outcome-based measures of assessing performance (e.g., speed and accuracy), but also addresses measures of operator state. Such measures include those obtained from subjective workload and fatigue probes, situation awareness (SA) probes, and continuous measures from eye tracking systems. The chapter closes by discussing a new testbed developed by the authors that enables the assessment of human-automation interaction across a broad range of mission contexts.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Beatty J, Lucero-Wagoner B (2000) The pupillary system. Handbook of psychophysiology 2:142-162

    Google Scholar 

  • Button K (2009) Different courses-New-style UAV trainees edge toward combat. C4isr 8 (10):34

    Google Scholar 

  • Caffier PP, Erdmann U, Ullsperger P (2003) Experimental evaluation of eye-blink parameters as a drowsiness measure. European journal of applied physiology 89 (3-4):319-325

    Article  Google Scholar 

  • Calhoun GL, Draper MH, Ruff HA Effect of level of automation on unmanned aerial vehicle routing task. In: Proceedings of the Human Factors and Ergonomics Society 53rd Annual Meeting, San Antonio, TX, 2009. vol 4. SAGE Publications, pp 197-201

    Google Scholar 

  • Calhoun GL, Ruff HA, Draper MH, Wright EJ (2011) Automation-level transference effects in simulated multiple unmanned aerial vehicle control. Journal of Cognitive Engineering and Decision Making 5 (1):55-82

    Article  Google Scholar 

  • Chanda M, DiPlacido J, Dougherty J, Egan R, Kelly J, Kingery T, Liston D, Mousseau D, Nadeau J, Rothman T, Smith L, Supko M (2010) Proposed functional architecture and associated benefits analysis of a common ground control station for Unmanned Aircraft Systems.

    Google Scholar 

  • Chen JY, Barnes MJ, Harper-Sciarini M (2011) Supervisory control of multiple robots: Human-performance issues and user-interface design. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 41 (4):435-454

    Article  Google Scholar 

  • Coyne J, Sibley C Investigating the Use of Two Low Cost Eye Tracking Systems for Detecting Pupillary Response to Changes in Mental Workload. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2016. vol 1. SAGE Publications, pp 37-41

    Google Scholar 

  • Cummings ML, Bertucelli LF, Macbeth J, Surana A (2014) Task versus vehicle-based control paradigms in multiple unmanned vehicle supervision by a single operator. IEEE Transactions on Human-Machine Systems 44 (3):353-361

    Article  Google Scholar 

  • Cummings ML, Nehme CE Modeling the impact of workload in network centric supervisory control settings. In: 2nd Annual Sustaining Performance Under Stress Symposium, 2009.

    Google Scholar 

  • Defense Science Board (2016) Summer Study on Autonomy. Office of the Under Secretary of Defense for Acquisition, Technology and Logistics, Washington, D.C.

    Google Scholar 

  • Department of Defense (2013) Unmanned systems integrated roadmap: FY2013-2038. Washington, DC, USA

    Google Scholar 

  • Durso FT, Dattel AR (2004) SPAM: The real-time assessment of SA. In: Banbury S, Tremblay S (eds) A cognitive approach to situation awareness: Theory and application, vol 1. Ashgate Publishing Ltd, Hampshire, pp 137-154

    Google Scholar 

  • Endsley MR Situation awareness global assessment technique (SAGAT). In: Proceedings of the National Aerospace and Electronics Conference, New York, 1988. IEEE, pp 789-795

    Google Scholar 

  • Endsley MR, Kaber DB (1999) Level of automation effects on performance, situation awareness and workload in a dynamic control task. Ergonomics 42 (3):462-492. doi:10.1080/001401399185595

    Article  Google Scholar 

  • Funke G, Greenlee E, Carter M, Dukes A, Brown R, Menke L Which Eye Tracker Is Right for Your Research? Performance Evaluation of Several Cost Variant Eye Trackers. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2016. vol 1. SAGE Publications, pp 1240-1244

    Google Scholar 

  • Gertler J (2012) US Unmanned Aerial Systems. Congressional Research Service,

    Google Scholar 

  • Giese S, Carr D, Chahl J Implications for unmanned systems research of military UAV mishap statistics. In: Intelligent Vehicles Symposium (IV), 2013 IEEE, 2013. IEEE, pp 1191-1196

    Google Scholar 

  • Holmqvist K, Nyström M, Andersson R, Dewhurst R, Jarodzka H, Van de Weijer J (2011) Eye tracking: A comprehensive guide to methods and measures. Oxford University Press, New York

    Google Scholar 

  • Holmqvist K, Nyström M, Mulvey F Eye tracker data quality: What it is and how to measure it. In: ACM, Proceedings of the Symposium on Eye Tracking Research and Applications, Santa Barbara, CA, 2012. pp 45-52

    Google Scholar 

  • Johnson R, Leen M, Goldberg D (2007) Testing adaptive levels of automation (ALOA) for UAV supervisory control (Technical Report AFRL-HE-WP-TR-2007-0068). Air Force Research Laboratory

    Google Scholar 

  • Kahneman D (1973) Attention and effort. Prentice-Hall, Kahneman, Daniel. Attention and effort. Englewood Cliffs, NJ

    Google Scholar 

  • Kidwell B, Calhoun GL, Ruff HA, Parasuraman R Adaptable and adaptive automation for supervisory control of multiple autonomous vehicles. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2012. vol 1. SAGE Publications, pp 428-432

    Google Scholar 

  • NATO (2012) Standard Interfaces of UAV Control System (UCS) for NATO UAV Interoperability. NATO Standardization Agency, Brussels, Belgium

    Google Scholar 

  • Nehme CE (2009) Modeling human supervisory control in heterogeneous unmanned vehicle systems. Massachusetts Institute of Technology, Cambridge, MA

    Google Scholar 

  • Office of Naval Research (2015) Naval Science & Technology Strategy. Department of the Navy, Arlington, VA

    Google Scholar 

  • Office of the Secretary of Defense (2012) Unmanned Aircraft Systems Ground Control Station Human-Machine Interface: Development and Standardization Guide. Washington, DC

    Google Scholar 

  • Ooms K, Dupont L, Lapon L, Popelka S (2015) Accuracy and precision of fixation locations recorded with the low-cost Eye Tribe tracker in different experimental set-ups. Journal of Eye Movement Research 8:1-24

    Google Scholar 

  • Parasuraman R, Manzey DH (2010) Complacency and bias in human use of automation: An attentional integration. Human Factors: The Journal of the Human Factors and Ergonomics Society 52 (3):381-410

    Article  Google Scholar 

  • Parasuraman R, Riley V (1997) Humans and automation: Use, misuse, disuse, abuse. Human Factors 39:230-253

    Article  Google Scholar 

  • Parasuraman R, Sheridan TB, Wickens CD (2000) A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans 30 (3):286-297. doi:10.1109/3468.844354

    Article  Google Scholar 

  • Parasuraman R, Sheridan TB, Wickens CD (2008) Situation awareness, mental workload, and trust in automation: Viable, empirically supported cognitive engineering constructs. Journal of Cognitive Engineering and Decision Making 2 (2):140-160

    Article  Google Scholar 

  • Ratwani RM, McCurry JM, Trafton JG Single operator, multiple robots: an eye movement based theoretic model of operator situation awareness. In: Proceedings of the 5th ACM/IEEE international conference on Human-robot interaction, 2010. IEEE Press, pp 235-242

    Google Scholar 

  • Samn SW, Perelli LP (1982) Estimating aircrew fatigue: a technique with application to airlift operations. DTIC Document, USAF School of Aerospace Medicine, Brooks Air Force Base, TX

    Google Scholar 

  • Sheridan T, Verplank W (1978) Human and computer control of undersea teleoperators. MIT Man-Machine Systems Laboratory, Cambridge, MA

    Google Scholar 

  • Sheridan TB (2000) Function allocation: algorithm, alchemy or apostasy? International Journal of Human-Computer Studies 52 (2):203-216

    Article  Google Scholar 

  • Sheridan TB (2012) Human Supervisory Control. In: Handbook of Human Factors and Ergonomics. John Wiley & Sons, Inc., pp 990-1015. doi:10.1002/9781118131350.ch34

  • Sibley C, Coyne J, Avvari GV, Mishra M, Pattipati KR (2016a) Supporting Multi-objective Decision Making Within a Supervisory Control Environment. In: Schmorrow DD, Fidopiastis CM (eds) Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience: 10th International Conference, AC 2016, Held as Part of HCI International 2016, Toronto, ON, Canada, July 17-22, 2016, Proceedings, Part II. Springer International Publishing, Cham, pp 210-221. doi:10.1007/978-3-319-39952-2_21

    Google Scholar 

  • Sibley C, Coyne J, Baldwin C Pupil dilation as an index of learning. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2011. vol 1. pp 237-241

    Google Scholar 

  • Sibley C, Coyne J, Thomas J Demonstrating the Supervisory Control Operations User Testbed (SCOUT). In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2016b. vol 1. SAGE Publications, pp 1324-1328

    Google Scholar 

  • Tsai Y-F, Viirre E, Strychacz C, Chase B, Jung T-P (2007) Task performance and eye activity: predicting behavior relating to cognitive workload. Aviation, space, and environmental medicine 78 (Supplement 1):B176-B185

    Google Scholar 

  • van de Merwe K, van Dijk H, Zon R (2012) Eye movements as an indicator of situation awareness in a flight simulator experiment. The International Journal of Aviation Psychology 22 (1):78-95

    Article  Google Scholar 

  • Williams KW (2004) A summary of unmanned aircraft accident/incident data: Human factors implications. DTIC Document, OKLAHOMA CITY, OK

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Naval Research Laboratory, SW, Washington, DC, USA

    Ciara Sibley & Joseph Coyne

  2. Embry-Riddle Aeronautical University, Daytona Beach, FL, USA

    Sarah Sherwood

Authors
  1. Ciara Sibley
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joseph Coyne
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sarah Sherwood
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ciara Sibley .

Editor information

Editors and Affiliations

  1. Paine College, Augusta, Georgia, USA

    W.F. Lawless

  2. Naval Research Laboratory, Washington, District of Columbia, USA

    Ranjeev Mittu

  3. Naval Research Laboratory, Washington, District of Columbia, USA

    Donald Sofge

  4. U.S. Army Research Laboratory, Adelphi, Maryland, USA

    Stephen Russell

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Sibley, C., Coyne, J., Sherwood, S. (2017). Research Considerations and Tools for Evaluating Human-Automation Interaction with Future Unmanned Systems. In: Lawless, W., Mittu, R., Sofge, D., Russell, S. (eds) Autonomy and Artificial Intelligence: A Threat or Savior?. Springer, Cham. https://doi.org/10.1007/978-3-319-59719-5_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-59719-5_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-59718-8

  • Online ISBN: 978-3-319-59719-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation