Scientific and Technical Information Processing

, Volume 44, Issue 5, pp 379–385 | Cite as

A Combined System for Contactless Control of Robotic Systems by Verbal and Gesture Commands

  • V. E. Nagapetyan
  • V. M. Khachumov


This article considers the task of building a universal integrated intelligent system and its interface for speech and gesture-activated control of robotic systems, including means of transport and aircraft. Several foreign publications are reviewed that present combined control systems and their results. The system is integrated with a knowledge base and a data base, makes use of artificial-intelligence elements, and bears on the generality of procedures of inputting, processing, and transferring different kinds of information.


command interface integration speech phoneme gesture image scan informative parameters recognition control 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bhuiyan, M. and Picking, R., Gesture-Controlled User Interfaces, What Have We Done and What’s Next?, Centre for Applied Internet Research (CAIR), Glyndŵr University, Wrexham, UK. http://www.glyndwr. Cited February 13, 2015.Google Scholar
  2. 2.
    Tasevski, J., Nikolić, M., and Mišković, D., Integration of an industrial robot with the systems for image and voice recognition, Serb. J. Electr. Eng., 2013, pp. 1–13. ms_for_Image_and_Voice_Recognition. Cited February 13, 2015.Google Scholar
  3. 3.
    Fardana, A.R., Jain, S., Jovancevic, I., Suri, Y., Morand, C., and Robertson, N.M., Controlling a Mobile Robot with Natural Commands based on Voice and Gesture. pdf. Cited February 13, 2015.Google Scholar
  4. 4.
    Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., and Blake, A., real-time human pose recognition in parts from single depth images computer vision and pattern recognition, IEEE Conference on Computer Vision and Pattern Recognition (20–25 June 2011), 2013. com/pubs/145347/bodypartrecognition.pdf. Cited February 13, 2015.Google Scholar
  5. 5.
    Urban, M. and Bajcsy, P., Fusion of Voice, Gesture, and Human-Computer Interface Controls for Remotely Operated Robot, Champaign, Illinois: National Center for Supercomputing Applications (NCSA), University of Illinois at Urbana-Champaign. http://isda.ncsa.illinois. edu/peter/publications/conferences/2005/Fusion 05Robot.pdf. Cited February 13, 2015.Google Scholar
  6. 6.
    Kotomin, A.V., Preprocessing of a sound signal in the system of recognition of speech commands, Trudy XV Molodezhnoi nauchnoi konferentsii “Naukoemkie informatsionnye tekhnologii” SIT-2011 (20–24 aprelya 2011 g.) (Proceedings of the XV Youth Scientific Conference High-Tech Information Technologies SIT-2011 (April 20–24, 2011)), Pereslavl’-Zalesskii: Izd. “Universitet goroda Pereslavlya,” 2011, pp. 25–38.Google Scholar
  7. 7.
    Mokrov, E.V., Khachumov, V.M., and Teploukhova, N.E., Application of the phonemic approach to the recognition of speech commands, in Raketno-kosmicheskoe priborostroenie i informatsionnye tekhnologii. 2011, Trudy IV Vserossiiskoi nauchno-tekhnicheskoi konferentsii “Aktual’nye problemy raketno-kosmicheskogo priborostroeniya i informatsionnykh tekhnologii” (15–17 iyunya 2011 g.) (Rocket and Space Instrument Making and Information Technologies. 2011. Proceedings of the IV All-Russian Scientific and Technical Conference “Actual Problems of Missile and Space Instrument-Making and Information Technologies” (June 15–17, 2011)), Moscow: Radiotekhnika, 2012, pp. 418–424.Google Scholar
  8. 8.
    Nahapetyan, V.E. and Khachumov, V.M., Automatic transformation of Russian manual alphabet gestures into textual form, Sci. Tech. Inf. Process., 2014, vol. 41, no. 5, pp. 302–308.CrossRefGoogle Scholar
  9. 9.
    Nagapetyan, V.E. and Khachumov, V.M., Recognition of hand gestures in the task of contactless control of an unmanned aerial vehicle, Avtometriya, 2015, no. 2.Google Scholar
  10. 10.
    Khachumov, M.V., Models of the representation and clustering of semistructured information, Sci. Tech. Inf. Process., 2015, vol. 42, no. 5, pp. 388–396.CrossRefGoogle Scholar
  11. 11.
    Talalaev, A.A. and Fralenko, V.P., A set of tools for the design of neural network applications, Nauchno-Tekh. Vestn. Povolzh’ya, 2013, no. 4, pp. 237–243.Google Scholar

Copyright information

© Allerton Press, Inc. 2017

Authors and Affiliations

  1. 1.Peoples’ Friendship University of RussiaMoscowRussia

Personalised recommendations