Playbook for UAS: UX of Goal-Oriented Planning and Execution

  • Jack GaleEmail author
  • John Karasinski
  • Steve Hillenius
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10906)


We are evaluating Playbook for CASAS (Connected Autonomous Smart Aerospace Systems), a tool designed to aid first responders in disaster relief efforts. We are adapting an existing tool, Playbook, to support a future unmanned aircraft system (UAS) swarm demonstration. Playbook for CASAS will be used to plan, edit, and monitor simulated UAS swarms, and we are interested in evaluating the user experience of this prototype as well as developing recommendations for future UAS interfaces. Allocation of roles and responsibilities between human-automation systems is key to promoting productive cooperation between users and automation. Future interfaces, however, must allow for adaptive management of the swarm not a constant split in human-automation control. Our early research indicates that when a single pilot is controlling swarms of robotic agents, such as UAS or ground rovers, operators require a higher level, goal-based interface with usability at its core. Along with that high-level control, users can leverage sensors within the swarm to be notified when lower level actions must be taken by the pilot. First responders working in disaster relief efforts require a high level of situational awareness (SA) and precise control at key moments within a mission. This balance in operator workload paired with SA can lead to improved safety and mission outcomes. Our research below outlines leverage points as well as the balance between human involvement and autonomy in UAS interfaces.


Swarms UAV UAS Autonomy Engineering psychology Cognitive ergonomics User experience Human centered design HCI 



The authors would like to thank Jimin Zheng, Matthew Chan, and Richard Joyce for support and development work during the various stages of this project. The authors would also like to thank the subject testing participants, Ames DART, as well as Eric Mueller for his leadership on the Connected Smart Aerospace Systems project (CASAS). This work was performed under a US Govt. Contract in the Human-Systems Integration Division at NASA Ames Research Center.


  1. 1.
    Endsley, M.R.: Toward a theory of situation awareness in dynamic systems. Hum. Fact. 37(1), 32–64 (1995)CrossRefGoogle Scholar
  2. 2.
    Barber, K.S., Han, D.C.: Multi-agent planning under dynamic adaptive autonomy. In: International Conference on Systems Man and Cybernetics. Institute of Electrical Engineers Inc. (IEEE) (1998)Google Scholar
  3. 3.
    Jung, D., Ratti, J., Tsiotras, P.: Real-time implementation and validation of a new hierarchical path planning scheme of UAVs via hardware-in-the-loop simulation. J. Intell. Rob. Syst. 54, 163 (2009)CrossRefGoogle Scholar
  4. 4.
    Rasche, C., et al.: Combining autonomous exploration, goal-oriented coordination and task allocation in multi-UAV scenarios. In: 2010 Sixth International Conference on Autonomic and Autonomous Systems (ICAS). IEEE (2010)Google Scholar
  5. 5.
    Hillenius, S.: Designing interfaces for astronaut autonomy in space. In: CanUX 2015, Ottawa, Canada (2015). (Invited Speech)Google Scholar
  6. 6.
    Hillenius, S., Marquez, J., Deliz, I., Kanefsky, B., Korth, D., Healy, M., Gibson, S., Zheng, J.: Designing and building a crew-centric mobile scheduling and planning tool for exploring crew autonomy concepts onboard the international space station. In: ISS R&D Conference 2016, San Diego, CA (2016). (Speech)Google Scholar
  7. 7.
    Space Shuttle Program Flight Data File. Accessed 9 Feb 2018
  8. 8.
    Apollo 11 Flight Plan. Accessed 9 Feb 2018
  9. 9.
    Hashemi, S., Hillenius, S.: @NASA: the user experience of a space station. In: SXSW Interactive 2013, Austin, TX (2013)Google Scholar
  10. 10.
    Erzberger, H.: Automated conflict resolution for air traffic control. In: Proceedings of 25th International Congress of the Aeronautical Sciences (ICAS), Germany (2006)Google Scholar
  11. 11.
    Ericsson, K.A., Simon, A.H.: Protocol Analysis: Verbal Reports as Data. MIT Press, Cambridge (1993)Google Scholar
  12. 12.
    NASA: HCI-05.: We need verifiable requirements that specify standard measurement techniques and metrics for evaluating the quality of user interfaces (2017). Accessed 7 Feb 2018
  13. 13.
    DART Homepage. Accessed 9 Feb 2018
  14. 14.
    Palat, R.C., Annamalau, A., Reed, J.R.: Cooperative relaying for ad-hoc ground net-works using swarm UAVs. In: Military Communications Conference, MILCOM 2005. IEEE, Atlantic City (2005)Google Scholar
  15. 15.
    Prevot, T., Homola, J., Mercer, J.: Human-in-the-loop evaluation of ground-based automated separation assurance for NextGen. In: 26th Congress of International Council of the Aeronautical Sciences (ICAS), Anchorage, Alaska, USA (2008)Google Scholar
  16. 16.
    NASA: HARI-02 We need to develop design guidelines for effective human-automation-robotic systems (2017). Accessed 9 Feb 2018
  17. 17.
    Lieberman, H., Espinosa, J.: A goal-oriented interface to consumer electronics using planning and commonsense reasoning. In: Proceedings of 11th International Conference on Intelligent User Interfaces - IUI (2007)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.San Jose State University Research FoundationSan JoseUSA
  2. 2.NASA Ames Research CenterMountain ViewUSA

Personalised recommendations