Skip to main content
SpringerLink
Log in

Exploring User-Defined Gestures and Voice Commands to Control an Unmanned Aerial Vehicle

  • Conference paper
  • First Online:
Intelligent Technologies for Interactive Entertainment (INTETAIN 2016 2016)

Abstract

In this paper we follow a participatory design approach to explore what novice users find to be intuitive ways to control an Unmanned Aerial Vehicle (UAV). We gather users’ suggestions for suitable voice and gesture commands through an online survey and a video interview and we also record the voice commands and gestures used by participants’ in a Wizard of Oz experiment where participants thought they were manoeuvring a UAV. We identify commonalities in the data collected from the three elicitation methods and assemble a collection of voice and gesture command sets for navigating a UAV. Furthermore, to obtain a deeper understanding of why our participants chose the gestures and voice commands they did, we analyse and discuss the collected data in terms of mental models and identify three prevailing classes of mental models that likely guided many of our participants in their choice of voice and gesture commands.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Please contact the authors for the full collection of voice and gesture commands.

References

  1. Supimros, S., Wongthanavasu, S.: Speech recognition - based control system for drone. In: Proceedings of the 3rd ICT International Student Project Conference (ICT-ISPC 2014), pp. 107–110 (2014)

    Google Scholar 

  2. Higuchi, K., Rekimoto, J.: Flying head: a head motion synchronization mechanism for unmanned aerial vehicle control. In: CHI 2013 Extended Abstracts on Human Factors in Computing Systems, pp. 2029–2038 (2013)

    Google Scholar 

  3. Teixeira, J.M., Ferreira, R., Santos, M., Teichrieb, V.: Teleoperation using google glass and AR.Drone for structural inspection. In: Proceedings of the XVI Symposium on Virtual and Augmented Reality (SVR 2014), pp. 28–36 (2014)

    Google Scholar 

  4. Pittman, C., LaViola, J.J.: Exploring head tracked head mounted displays for first person robot teleoperation. In: Proceedings of the 2014 ACM International Conference on Intelligent User Interfaces (IUI 2014), pp. 323–328 (2014)

    Google Scholar 

  5. Mashood, A., Noura, H., Jawhar, I., Mohamed, N.: A gesture based kinect for quadrotor control. In: Proceedings of the 2015 International Conference on Information and Communication Technology Research (ICTRC 2015), pp. 298–301 (2015)

    Google Scholar 

  6. Naseer, T., Sturm, J., Cremers, D.: FollowMe: person following and gesture recognition with a quadrocopter. In: Proceedings of the 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2013), pp. 624–630 (2013)

    Google Scholar 

  7. Ng, W.S., Sharlin, E.: Collocated interaction with flying robots. In: Proceedings of the 20th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2011), pp. 143–149 (2011)

    Google Scholar 

  8. Soto-Guerrero, D., Ramírez Torres, J.G.: A human-machine interface with unmanned aerial vehicles. In: Proceedings of the 10th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE 2013), pp. 307–312 (2013)

    Google Scholar 

  9. Pfeil, K., Koh, S.L., LaViola, J.: Exploring 3d gesture metaphors for interaction with unmanned aerial vehicles. In: Proceedings of the International Conference on Intelligent User Interfaces (IUI 2013), pp. 257–266 (2013)

    Google Scholar 

  10. Ikeuchi, K., Otsuka, T., Yoshii, A., Sakamoto, M., Nakajima, T.: KinecDrone: enhancing somatic sensation to fly in the sky with kinect and AR.Drone. In: Proceedings of the 5th Augmented Human International Conference (AH 2014), no. 53 (2014)

    Google Scholar 

  11. Nagi, J., Giusti, A., Gambardella, L.M., Di Caro, G.A.: Human-swarm interaction using spatial gestures. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014), pp. 3834–3841 (2014)

    Google Scholar 

  12. Monajjemi, V., Wawerla, J., Vaughan, R., Mori, G.: HRI in the sky: creating and commanding teams of UAVs with a vision-mediated gestural interface. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2013), pp. 617–623 (2013)

    Google Scholar 

  13. Hansen, J.P., Alapetite, A., MacKenzie, I.S., Møllenbach, E.: The use of gaze to control drones. In: Proceedings of the ACM Symposium on Eye Tracking Research and Applications (ETRA 2014), pp. 27–34 (2014)

    Google Scholar 

  14. Kos’myna, N., Tarpin-Bernard, F., Rivet, B.: Bidirectional feedback in motor imagery BCIs: learn to control a drone within 5 minutes. In: CHI 2014 Extended Abstracts on Human Factors in Computing Systems, pp. 479–482 (2014)

    Google Scholar 

  15. Davidson, M.J., Dove, L., Weltz, J.: Mental Models and Usability (1999)

    Google Scholar 

  16. Rust, K., Malu, M., Anthony, L., Findlater, L.K.: Understanding child-defined gestures and children’s mental models for touchscreen tabletop interaction. In: Proceedings of the 2014 Conference on Interaction Design and Children (IDC 2014), pp. 201–204 (2014)

    Google Scholar 

  17. Valdes, C., Eastman, D., Grote, C., Thatte, S., Shaer, O., Mazalek, A., Ullmer, B., Konkel, M.K.: Exploring the design space of gestural interaction with active tokens through user-defined gestures. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2014), pp. 4107–4116 (2014)

    Google Scholar 

  18. Kiesler, S., Goetz, J.: Mental models of robotic assistants. In: CHI 2002 Extended Abstracts on Human Factors in Computing Systems, pp. 576–577 (2002)

    Google Scholar 

  19. Powers, A., Kiesler, S.: The advisor robot: tracing people’s mental model from a robot’s physical attributes. In: Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction (HRI 2006), pp. 218–225 (2006)

    Google Scholar 

  20. Fong, T., Nourbakhsh, I., Dautenhahn, K.: A survey of socially interactive robots. Robot. Auton. Syst. 42(3–4), 143–166 (2003)

    Article  MATH  Google Scholar 

  21. Hauptmann, A., McAvinney, P.: Gestures with speech for graphic manipulation. Int. J. Man-Mach. Stud. 38(2), 231–249 (1993)

    Article  Google Scholar 

  22. Sharma, R., Pavlović, V.I., Huang, T.: Toward multimodal human-computer interface. Proc. IEEE 86(5), 853–869 (1998)

    Article  Google Scholar 

  23. Rogalla, O., Ehrenmann, M., Zöllner, R., Becher, R., Dillmann, R.: Using gesture and speech control for commanding a robot assistant. In: Proceedings of the 11th International Workshop on Robot and Human Interactive Communication (RO-MAN 2002), pp. 454–459 (2002)

    Google Scholar 

  24. Urban, M., Bajcsy, P.: Fusion of voice, gesture, and human-computer controls for remotely operated robot. In: Proceedings of the 7th International Conference on Information Fusion (FUSION 2005), pp. 1644–1651 (2005)

    Google Scholar 

  25. Jones, G., Berthouze, N., Bielski, R., Julier, S.: Towards a situated, multimodal interface for multiple UAV control. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2010), pp. 1739–1744 (2010)

    Google Scholar 

  26. Quigley, M., Goodrich, M.A., Beard, R.W.: Semi-autonomous human-UAV interfaces for fixed-wing mini-UAVs. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2004), pp. 2457–2462 (2004)

    Google Scholar 

  27. Lichtenstern, M., Frassl, M., Perun, B., Angermann, M.: A prototyping environment for interaction between a human and a robotic multi-agent system. In: Proceedings of the 7th Annual ACM/IEEE International Conference on Human-Robot Interaction (HRI 2012), pp. 185–186 (2012)

    Google Scholar 

  28. Burke, M., Lasenby, J.: Pantomimic gestures for human–robot interaction. IEEE Trans. Robot. 31(5), 1225–1237 (2015)

    Article  Google Scholar 

  29. Cauchard, J.R., E, J.L., Zhai, K.Y., Landay, J.A.: Drone & me: an exploration into natural human-drone interaction. In: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2015), pp. 361–365 (2015)

    Google Scholar 

  30. Craik, K.: The Nature of Explanation. Cambridge University Press, Cambridge (1943)

    Google Scholar 

  31. Norman, D.A.: The Design of Everyday Things. Basic Books, Inc., New York (2002)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Alpen-Adria-Universität Klagenfurt, Klagenfurt, Austria

    Ekaterina Peshkova, Martin Hitz & David Ahlström

Authors
  1. Ekaterina Peshkova
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martin Hitz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David Ahlström
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ekaterina Peshkova .

Editor information

Editors and Affiliations

  1. Department of Information and Computing Sciences, Utrecht University, Utrecht, The Netherlands

    Ronald Poppe

  2. Department of Information and Computing Sciences, Utrecht University, Utrecht, The Netherlands

    John-Jules Meyer

  3. Department of Information and Computing Sciences, Utrecht University, Utrecht, The Netherlands

    Remco Veltkamp

  4. Department of Information and Computing Sciences, Utrecht University, Utrecht, The Netherlands

    Mehdi Dastani

Rights and permissions

Reprints and permissions

Copyright information

© 2017 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Cite this paper

Peshkova, E., Hitz, M., Ahlström, D. (2017). Exploring User-Defined Gestures and Voice Commands to Control an Unmanned Aerial Vehicle. In: Poppe, R., Meyer, JJ., Veltkamp, R., Dastani, M. (eds) Intelligent Technologies for Interactive Entertainment. INTETAIN 2016 2016. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 178. Springer, Cham. https://doi.org/10.1007/978-3-319-49616-0_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-49616-0_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-49615-3

  • Online ISBN: 978-3-319-49616-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation