Skip to main content
SpringerLink
Log in

A Natural Interaction Interface for UAVs Using Intuitive Gesture Recognition

  • Conference paper
  • First Online:
Book cover Advances in Human Factors in Robots and Unmanned Systems

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 499))

Abstract

The popularity of unmanned aerial vehicles (UAVs) is increasing as technological advancements boost their favorability for a broad range of applications. One application is science data collection. In fields like earth and atmospheric science, researchers are seeking to use UAVs to augment their current portfolio of platforms and increase their accessibility to geographic areas of interest. By increasing the number of data collection platforms, UAVs will significantly improve system robustness and allow for more sophisticated studies. Scientists would like the ability to deploy an available fleet of UAVs to traverse a desired flight path and collect sensor data without needing to understand the complex low-level controls required to describe and coordinate such a mission. A natural interaction interface for a Ground Control System (GCS) using gesture recognition is developed to allow non-expert users (e.g., scientists) to define a complex flight path for a UAV using intuitive hand gesture inputs from the constructed gesture library. The GCS calculates the combined trajectory on-line, verifies the trajectory with the user, and sends it to the UAV controller to be flown.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Jenkins, D., Bijan, B.: The economic impact of unmanned aircraft systems integration in the United States. Technical Report, AUVSI (2013)

    Google Scholar 

  2. Wald, M.L.: Domestic drones stir imaginations and concerns. New York Times (2013)

    Google Scholar 

  3. Barr, A.: Amazon testing delivery by drone, CEO Bezos Says. USA Today. http://www.usatoday.com/story/tech/2013/12/01/amazon-bezos-dronedelivery/3799021/

  4. Saggiani, G., Teodorani, B.: Rotary wing UAV potential applications: an analytical study through a matrix method. Aircraft Eng. Aerosp. Technol. 76(1), 6–14 (2004)

    Article  Google Scholar 

  5. Office of the Secretary of Defense.: Unmanned aircraft systems roadmap 2005–2030. Washington, DC (2005)

    Google Scholar 

  6. Schoenwald, D.A.: AUVs: in space, air, water, and on the ground. IEEE Control Syst. Mag. 20(6), 15–18 (2000)

    Google Scholar 

  7. Chen, H., Wang, X., Li, Y.: A Survey of autonomous control for UAV. In: IEEE International Conference on Artificial Intelligence and Computational Intelligence, pp. 267–271. Shanghai (2009)

    Google Scholar 

  8. Hsu, J.: MIT researcher develops iPhone app to easily control swarms of aerial drones. In: Popular Science. http://www.popsci.com/technology/article/2010-04/video-usingsmartphonescontrol-aerial-drones

  9. Zelinski, S., Koo, T.J., Sastry, S.: Hybrid system design for formations of autonomous vehicles. In: IEEE Conference on Decision and Control (2003)

    Google Scholar 

  10. Miniature Air Quality Monitoring System, http://www.altechusa.com/odor.html

  11. Wegener, S., Schoenung, et al.: UAV autonomous operations for airborne science missions. In: AIAA 3rd Unmanned Unlimited Technical Conference, Chicago (2004)

    Google Scholar 

  12. Wegener, S., Schoenung, S.: Lessons learned from NASA UAV science demonstration program missions. In: AIAA 2nd Unmanned Unlimited Systems, Technologies and Operations Conference, San Diego (2003)

    Google Scholar 

  13. Perzanowski, D., Schultz, A.C., et al.: Building a multimodal human-robot interaction. In: IEEE Intelligent Systems, pp. 16–21 (2001)

    Google Scholar 

  14. Trujillo, A.C., Fan, H., et al.: Operator informational needs for multiple autonomous small vehicles. In: 6th International Conference on Applied Human Factors and Ergonomics and the Affiliated Conferences, Las Vegas, pp. 5556–5563 (2015)

    Google Scholar 

  15. Trujillo, A.C., Cross, C., et al.: Collaborating with autonomous agents. In: 15th AIAA Aviation Technology, Integration and Operations Conference, Dallas (2015)

    Google Scholar 

  16. Reitsema, J., Chun, W., et al.: Team-centered virtual interaction presence for adjustable autonomy. In: AIAA Space, Long Beach (2005)

    Google Scholar 

  17. Wachs, J. P., K¨olsch, M., Stern, H., Edan, Y.: Vision-based hand-gesture applications. Commun. ACM 54(2), 60–71 (2011)

    Google Scholar 

  18. McCafferty, S.: Space for cognition: gesture and second language learning. Int. J. Appl. Linguist. 14(1), 148–165 (2004)

    Article  Google Scholar 

  19. Torrance, M.: Natural communication with robots. M.S. Thesis, Department of Electrical Engineering and Computer Science, MIT, Cambridge (1994)

    Google Scholar 

  20. Iba, S., Weghe, M.V., et al.: An architecture for gesture based control of mobile robots. In: IEEE International Conference on Intelligent Robots and Systems, pp. 851–857 (1999)

    Google Scholar 

  21. Neto, P., Pires, J.N.: High-level programming for industrial robotics: using gestures, speech and force control. In: IEEE International Conference on Robotics and Automation, Kobe (2009)

    Google Scholar 

  22. Yeo, Z.: GestureBots: intuitive control for small robots. In: CHI 28th Annual ACM Conference on Human Factors in Computing Systems, Atlanta (2010)

    Google Scholar 

  23. Waldherr, S., Romero, R., Thrun, S.: A gesture based interface for human-robot interaction. Auton. Robots 9, 151–173 (2000)

    Article  Google Scholar 

  24. Becker, M., Kefalea, E., et al.: GripSee: a gesture-controlled robot for object perception and manipulation. Auton. Robots 6, 203–221 (1999)

    Google Scholar 

  25. Lambrecht, J., Kleinsorge, M., Kruger, J.: Markerless gesture-based motion control and programming of industrial robots. In: 16th IEEE International Conference on Emerging Technologies and Factory Automation, Toulouse (2011)

    Google Scholar 

  26. Raheja, J.L., et al.: Real-time robotic hand control using hand gestures. In: IEEE 2nd International Conference on Machine Learning and Computing, Washington, DC (2010)

    Google Scholar 

  27. Bassily, D., et al.: Intuitive and adaptive robotic arm manipulation using the leap motion controller. In: 8th German Conference on Robotics, Munich (2014)

    Google Scholar 

  28. King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009)

    Google Scholar 

  29. Choe, R., et al.: Trajectory generation using spatial pythagorean hodograph Bézier curves. In: AIAA Guidance, Navigation and Control Conference, Kissimmee (2015)

    Google Scholar 

  30. DDS Standard, https://www.rti.com/products/dds/omg-ddsstandard.html

Download references

Acknowledgments

The authors would like to thank Javier Puig-Navarro and Syed Mehdi (NASA LaRC interns, University of Illinois at Urbana-Champaign) for their help in building the trajectory module, Gil Montague (NASA LaRC intern, Baldwin Wallace University) and Ben Kelley (NASA LaRC) for their expertise in integrating the DDS middleware, Dr. Loc Tran (NASA LaRC) for his insight on feature extraction, and the entire NASA LaRC Autonomy Incubator team for their invaluable support and feedback.

Author information

Authors and Affiliations

  1. Carnegie Mellon University, Pittsburgh, PA, USA

    Meghan Chandarana & Kenji Shimada

  2. NASA Langley Research Center, Hampton, VA, USA

    Anna Trujillo & B. Danette Allen

Authors
  1. Meghan Chandarana
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anna Trujillo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kenji Shimada
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. B. Danette Allen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meghan Chandarana .

Editor information

Editors and Affiliations

  1. U.S. Army Research Laboratory, Aberdeen Proving Ground, USA

    Pamela Savage-Knepshield

  2. U.S. Army Research Laboratory, Aberdeen Proving Ground, USA

    Jessie Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing Switzerland

About this paper

Cite this paper

Chandarana, M., Trujillo, A., Shimada, K., Danette Allen, B. (2017). A Natural Interaction Interface for UAVs Using Intuitive Gesture Recognition. In: Savage-Knepshield, P., Chen, J. (eds) Advances in Human Factors in Robots and Unmanned Systems. Advances in Intelligent Systems and Computing, vol 499. Springer, Cham. https://doi.org/10.1007/978-3-319-41959-6_32

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-41959-6_32

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-41958-9

  • Online ISBN: 978-3-319-41959-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation