Multi-UAV-Based Reconnaissance and Assessment of Helicopter Landing Points in Manned-Unmanned-Teaming Missions

Human-in-the-Loop Evaluation and Results
  • Marc SchmittEmail author
  • Peter Stuetz
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11472)


This article presents implementation aspects and experimental results for a cooperative multi-UAV team sensor and perception system. The presented on-board system is integrated in a R&D full-mission helicopter simulator used for manned-unmanned teaming (MUM-T) research in complex military search & rescue scenarios requiring field landings in uncontrolled and unsafe areas. Thus, to reduce the risk, the presented multi-UAV system is capable of performing highly-automated landing zone reconnaissance and landing point evaluation. Thereby, the presented sensor and perception management system (SPMS) incorporates knowledge on information demands regarding safe landing points and probabilistic reliability estimations of applied perceptive capabilities to derive its own course of actions for the landing zone reconnaissance. Expert knowledge is used to weight the single criteria in the multi-dimensional landing point assessment process while a perceptive trustworthiness estimation is used to handle the uncertainty of the measuring processes when fusing the reconnaissance results to assess the single landing points and to create a landing point recommendation for the helicopter crew. The presented system was evaluated in an extensive human-in-the-loop campaign with German Army Aviators. This paper presents questionnaire-based results gathered during the experimental campaign, focusing on human factors, user acceptance, and system design aspects.


MUM-T Manned-Unmanned-Teaming Multi-UAV Perception management Human factors Human-in-the-Loop 


  1. 1.
    NATO Standardization Agency: STANAG 2999 - Use of Helicopters in Land Operations Doctrine ATP-49 (F) (2012)Google Scholar
  2. 2.
    NATO Standardization Agency: STANAG 4586 - Standard Interface of UAV Control System (UCS) for NATO UAV Interoperability (2012)Google Scholar
  3. 3.
    Schulte, A., Donath, D., Honecker, F.: Human-system interaction analysis for military multi-RPA pilot activity and mental workload determination. In: IEEE International Conference on Systems, Man, and Cybernetics (SMC 2015), pp. 1375–1380 (2015)Google Scholar
  4. 4.
    Honecker, F., Schulte, A.: Automated online determination of pilot activity under uncertainty by using evidential reasoning. In: Harris, D. (ed.) EPCE 2017. LNCS (LNAI), vol. 10276, pp. 231–250. Springer, Cham (2017). Scholar
  5. 5.
    Brand, Y., Schulte, A.: Model-based prediction of workload for adaptive associate systems. In: 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC 2017), pp. 1722–1727. IEEE (2017)Google Scholar
  6. 6.
    Strenzke, R., Uhrmann, J., Benzler, A., Maiwald, F., Rauschert, A., Schulte, A.: Managing cockpit crew excess task load in military manned-unmanned teaming missions by dual-mode cognitive automation approaches. In: AIAA Guidance, Navigation, and Control Conference (2011)Google Scholar
  7. 7.
    Honecker, F., Brand, Y., Schulte, A.: A task-centered approach for workload-adaptive pilot associate systems. In: The 32nd EAAP Conference, Cascais, Portugal (2016)Google Scholar
  8. 8.
    Brand, Y., Schulte, A.: Design and evaluation of a workload-adaptive associate system for cockpit crews. In: Harris, D. (ed.) EPCE 2018. LNCS (LNAI), vol. 10906, pp. 3–18. Springer, Cham (2018). Scholar
  9. 9.
    Ruf, C., Stütz, P.: Model-driven sensor operation assistance for a transport helicopter crew in manned-unmanned teaming missions: selecting the automation level by machine decision-making. In: Savage-Knepshield, P., Chen, J. (eds.) Advances in Human Factors in Robots and Unmanned Systems. AISC, vol. 499, pp. 253–265. Springer, Cham (2017). Scholar
  10. 10.
    Ruf, C., Stütz, P.: Model-driven payload sensor operation assistance for a transport helicopter crew in manned-unmanned teaming missions: assistance realization, modelling experimental evaluation of mental workload. In: Harris, D. (ed.) EPCE 2017. LNCS (LNAI), vol. 10275, pp. 51–63. Springer, Cham (2017). Scholar
  11. 11.
    Ruf, C., Stütz, P.: Establishing a variable automation paradigm for UAV-based reconnaissance in manned-unmanned teaming missions. In: Chen, J. (ed.) AHFE 2018. AISC, vol. 784, pp. 24–35. Springer, Cham (2019). Scholar
  12. 12.
    Parasuraman, R., Sheridan, T.B., Wickens, C.D.: A model for types and levels of human interaction with automation. IEEE Trans. Syst. Man Cybern.-Part A: Syst. Hum. 30, 286–297 (2000)CrossRefGoogle Scholar
  13. 13.
    Sheridan, T.B.: Adaptive automation, level of automation, allocation authority, supervisory control, and adaptive control: distinctions and modes of adaptation. IEEE Trans. Syst. Man Cybern.-Part A: Syst. Hum. 41, 662–667 (2011)CrossRefGoogle Scholar
  14. 14.
    Schmitt, M., Stuetz, P.: Perception-oriented cooperation for multiple UAVs in a perception management framework: system concept and first results. In: 2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC). IEEE, Sacramento (2016)Google Scholar
  15. 15.
    U.S. Department of the Army: Attack Reconnaissance Helicopter Operations (FM 3-04.126) (2007).
  16. 16.
    Schmitt, M., Stütz, P.: Multi-UAV based helicopter landing zone reconnaissance. In: Harris, D. (ed.) EPCE 2017. LNCS (LNAI), vol. 10276, pp. 266–283. Springer, Cham (2017). Scholar
  17. 17.
    Russ, M., Stuetz, P.: Application of a probabilistic market-based approach in UAV sensor & perception management. In: 2013 16th International Conference on Information Fusion (FUSION 2013), Istanbul, pp. 676–683 (2013)Google Scholar
  18. 18.
    Hellert, C., Stuetz, P.: Performance prediction and selection of aerial perception functions during UAV missions. In: 2017 IEEE Aerospace Conference. IEEE, Big Sky (2017)Google Scholar
  19. 19.
    Russ, M., Schmitt, M., Hellert, C., Stuetz, P.: Airborne sensor and perception management: experiments and results for surveillance UAS. In: AIAA Infotech@Aerospace (I@A) Conference, Guidance, Navigation, and Control and Co-located Conferences, (AIAA 2013-5144), pp. 1–16. AIAA, Boston (2013)Google Scholar
  20. 20.
    Sheridan, T.B.: Telerobotics, Automation, and Human Supervisory Control. MIT Press, Cambridge (1992)Google Scholar
  21. 21.
    Hellert, C., Smirnov, D., Stuetz, P.: Ontologiedesign für Sensor- und Perzeptionsfähigkeiten von UAVs. In: Deutscher Luft- und Raumfahrtkongress 2014. Deutsche Gesellschaft für Luft- und Raumfahrt - Lilienthal-Oberth e.V., Augsburg (2014)Google Scholar
  22. 22.
    Smirnov, D., Stuetz, P.: Knowledge elicitation and representation for module based perceptual capabilities onboard UAVs. In: AIAA SciTech 2014. American Institute of Aeronautics and Astronautics, National Harbour (2014)Google Scholar
  23. 23.
    Smirnov, D., Stuetz, P.: Ontology-based modelling of sensor and data processing resources using OWL: a proof of concept. In: The First International Conference on Applications and Systems of Visual Paradigms. IARIA, Barcelona (2016)Google Scholar
  24. 24.
    Russ, M., Stuetz, P.: Airborne sensor and perception management: a conceptual approach for surveillance UAS. In: Proceedings of the 15th International Conference on Information Fusion (FUSION 2012), pp. 2444–2451. IEEE, Singapore (2012)Google Scholar
  25. 25.
    Schulte, A., Donath, D., Lange, D.S.: Design patterns for human-cognitive agent teaming. In: Harris, D. (ed.) EPCE 2016. LNCS (LNAI), vol. 9736, pp. 231–243. Springer, Cham (2016). Scholar
  26. 26.
    Uhrmann, J., Schulte, A.: Concept, design and evaluation of cognitive task-based UAV guidance. Int. J. Adv. Intell. Syst. 5, 145–158 (2012)Google Scholar
  27. 27.
    Meyer, C., Schulte, A.: Concept for satisfying pilot information demands in MUM-T under selective datalink availability. In: Proceedings of the 33rd European Association for Aviation Psychology Conference (EEAP33). European Association for Aviation Psychology, Dubrovnik (2018)Google Scholar
  28. 28.
    Hellert, C., Stuetz, P.: A concept for adaptation of perceptual capabilities for UAV platforms using case-based reasoning. In: AIAA SciTech 2014, 52nd Aerospace Sciences Meeting, pp. 1–8. American Institute of Aeronautics and Astronautics, National Harbour (2014)Google Scholar
  29. 29.
    U.S. Department of Defense: MIL-STD-2525C - Common Warfighting Symbology (2008).
  30. 30.
    Rudnick, G., Schulte, A.: Implementation of a responsive human automation interaction concept for task-based-guidance systems. In: Harris, D. (ed.) EPCE 2017. LNCS (LNAI), vol. 10275, pp. 394–405. Springer, Cham (2017). Scholar
  31. 31.
    Uhrmann, J., Strenzke, R., Schulte, A.: Task-based guidance of multiple detached unmanned sensor platforms in military helicopter operations. In: COGIS (COGnitive systems with Interactive Sensors), Crawley, UK (2010)Google Scholar
  32. 32.
    Billings, C.E.: Human-Centered Aviation Automation: Principles and Guidelines. NASA, Moffett Field (1996)Google Scholar
  33. 33.
    Seong, Y., Bisantz, A.M.: The impact of cognitive feedback on judgment performance and trust with decision aids. Int. J. Ind. Ergon. 38, 608–625 (2008)CrossRefGoogle Scholar
  34. 34.
    Likert, R.: A technique for the measurement of attittudes. Arch. Psychol. 140, 1–55 (1932)Google Scholar
  35. 35.
    Lee, J.D., Moray, N.: Trust, self-confidence, and operators’ adaptation to automation. Int. J. Hum Comput. Stud. 40, 153–184 (1994)CrossRefGoogle Scholar
  36. 36.
    Lee, J.D., See, K.A.: Trust in automation: designing for appropriate reliance. Hum. Factors J. Hum. Factors Ergon. Soc. 46, 50–80 (2004)CrossRefGoogle Scholar
  37. 37.
    Sharit, J.: Human error and human reliability analysis. In: Salvendy, G. (ed.) Handbook of Human Factors and Ergonomics, pp. 734–800. Wiley, Hoboken (2012)CrossRefGoogle Scholar
  38. 38.
    Cummings, M.L., Meyers, K., Scott, S.D.: Modified cooper harper evaluation tool for unmanned vehicle displays. In: Proceedings of UVS Canada: Conference on Unmanned Vehicle Systems Canada, Montebello, PQ, Canada (2006)Google Scholar
  39. 39.
    Donmez, B., Cummings, M.L., Graham, H.D., Brzezinski, A.S.: Modified cooper harper scales for assessing unmanned vehicle displays. In: Proceedings of the 10th Performance Metrics for Intelligent Systems Workshop on - PerMIS 2010, pp. 235–242. ACM Press, Baltimore (2010)Google Scholar
  40. 40.
    Donmez, B., Brzezinski, A.S., Graham, H.D., Cummings, M.L.: Modified Cooper Harper Scales for Assessing Unmanned Vehicle Displays. MIT Humans and Automation Laboratory, Cambridge (2008)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Institute of Flight Systems (IFS)University of the Bundeswehr Munich (UBM)NeubibergGermany

Personalised recommendations