Mobile Networks and Applications

, Volume 23, Issue 4, pp 797–805 | Cite as

Gesture Recognition Based on Kinect and sEMG Signal Fusion

  • Ying Sun
  • Cuiqiao Li
  • Gongfa LiEmail author
  • Guozhang Jiang
  • Du Jiang
  • Honghai Liu
  • Zhigao ZhengEmail author
  • Wanneng Shu


A weighted fusion method of D-S evidence theory in decision making is proposed to aim at the problem of lacking in the distribution of trust, data processing and precision in D-S evidential theory. The method of gesture recognition based on Kinect and sEMG signal are established. Weighted D-S evidence theory is used to fuse Kinect and sEMG signals and the simulation experiment is made respectively. The stimulation results show that comparing with other experimental methods, the decision fusion method based on weighted D-S evidence theory has higher utilization efficiency and recognition rate.


Gesture recognition D-S evidence theory sEMG Kinect signal fusion 



This work was supported by grants of National Natural Science Foundation of China (Grant No. 51575407, 51575338, 61273106, 51575412 and 61603420) and the Grants of National Defense Pre-Research Foundation of Wuhan University of Science and Technology (GF201705).


  1. 1.
    Guan R, Xu XM, Luo YY (2013) A computer vision-based gesture detection and recognition technique. Computer Applications and Software 30(1):155–159Google Scholar
  2. 2.
    Li J, Gu D, Liu F (2015) Finger and hand tracking based on kinect depth information. Computer Applications and Software 32(3):79–83Google Scholar
  3. 3.
    Fang YF, Liu HH, Li GF, Zhu XY (2015) A multichannel surface EMG system for hand motion recognition. International Journal of Humanoid Robotics 12(2):1–11CrossRefGoogle Scholar
  4. 4.
    Liu Y, Yang WL (2016) Hand gesture recognition based on Kinect and Pseudo-Jacobi-Fourier moments. Transducer and Microsystem Technologies 35(7):48–54Google Scholar
  5. 5.
    Lee HK, Shin SG, Kwon DS (2017) Design of emergency braking algorithm for pedestrian protection based on multi-sensor fusion. Int J Automot Technol 18(6):1067–1076CrossRefGoogle Scholar
  6. 6.
    Belmonte-Hernandez A, Hernandez-Penaloza G, Alvarez F, Conti G (2017) Adaptive Fingerprinting in Multi-Sensor Fusion for Accurate Indoor Tracking. IEEE Sensors J 17(15):4983–4998CrossRefGoogle Scholar
  7. 7.
    Das S, Barani S, Wagh S, Sonavane SS (2017) Extending lifetime of wireless sensor networks using multi-sensor data fusion. Sadhana-Academy Proceedings in Engineering Sciences 42(7):1083–1090Google Scholar
  8. 8.
    Fu H, Du XK (2005) Multi-sensor optimum fusion based on the bayes estimation. Control Theory and Applications 4(24):10–12Google Scholar
  9. 9.
    Wadaa A, Olariu S, Wilson L (2005) Training a wireless sensor network. Mobile Networks and Applications 10(1–2):151–168CrossRefGoogle Scholar
  10. 10.
    Yang P, Chen X, Li Y (2012) A sign language recognition method based on multi-sensor information. Space Medicine and Medical Engineering 25(4):276–281Google Scholar
  11. 11.
    Wang WH, Chen X, Yang P (2010) Chinese sign language recognition based on multiple sensors information detection and fusion. Chin J Biomed Eng 29(5):665–671Google Scholar
  12. 12.
    Zhao AF, Pei D, Wang QZ (2014) Gesture recognition fused with multi-information in complex environment. Computer Engineering and Applications 50(5):180–184Google Scholar
  13. 13.
    Chen DS, Li GF, Sun Y, Jiang GZ, Kong JY, Li JH, Liu HH (2017) Fusion hand gesture segmentation and extraction based on CMOS sensor and 3d sensor. Int J Wirel Mob Comput 12(3):305–312CrossRefGoogle Scholar
  14. 14.
    Ju ZJ, Liu HH (2011) A unified fuzzy framework for human-hand motion recognition. IEEE Trans Fuzzy Syst 19(5):901–913CrossRefGoogle Scholar
  15. 15.
    Ju ZJ, Liu HH (2010) Recognizing hand grasp and manipulation through empirical copula. Int J Soc Robot 2(3):321–328CrossRefGoogle Scholar
  16. 16.
    Chen DS, Li GF, Sun Y, Kong JY, Jiang GZ, Tang H, Ju ZJ, Yu H, Liu HH (2017) An interactive image segmentation method in hand gesture recognition. Sensors 17(2):253CrossRefGoogle Scholar
  17. 17.
    Liao YJ, Sun Y, Li GF, Kong JY, Jiang GZ, Jiang D, Cai HB, Ju ZJ, Yu H, Liu HH (2017) Simultaneous calibration: a joint optimization approach for multiple kinect and external cameras. Sensors 17(7):1491CrossRefGoogle Scholar
  18. 18.
    Miao W, Li GF, Jiang GZ, Fang YF, Ju ZJ, Liu HH (2015) Optimal grasp planning of multi-fingered robotic hands: a review. Applied and Computationnal Mathematics 14(3):238–247MathSciNetzbMATHGoogle Scholar
  19. 19.
    Krishna RV, Maziar L, Joohee K (2014) Real-time refinement of kinect depth maps using multi-resolution anisotropic diffusion. Mobile Networks and Applications 19(3):414–425CrossRefGoogle Scholar
  20. 20.
    Li GF, Liu J, Jiang GZ, Liu HH (2015) Numerical simulation of temperature field and thermal stress field in the new type of ladle with the nanometer adiabatic material. Advances in Mechanical Engineering 7(4):1–13Google Scholar
  21. 21.
    Li GF, Gu YS, Kong JY, Jiang GZ, Xie LX, Wu ZH, Li Z, He Y, Gao P (2013) Intelligent control of air compressor production process. Applied Mathematics & Information Sciences 7(3):1051–1058CrossRefGoogle Scholar
  22. 22.
    Alon J, Athitsos V (2009) A unified framework for gesture recognition and spatiotemporal gesture segmentation. IEEE Transactions on Pattem Analysis and Machine Intelligence 31(9):1685–1699CrossRefGoogle Scholar
  23. 23.
    Li GF, Qu PX, Kong JY, Jiang GZ, Xie LX, Gao P, Wu ZH, He Y (2013) Coke oven intelligent integrated control system. Applied Mathematics & Information Sciences 7(3):1043–1050CrossRefGoogle Scholar
  24. 24.
    Li GF, Miao W, Jiang GZ, Fang YF, Ju ZJ, Liu HH (2015) Intelligent control model and its simulation of flue temperature in coke oven. Discrete and Continuous Dynamical Systems - Series S (DCDS-S) 8(6):1223–1237MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Binh N, Enolkida S, Ejima T (2005). Real-time hand tracking and gesture recognition system. International Conference on Graphic, Vision and Image Processing, pp362–368Google Scholar
  26. 26.
    Li GF, Kong JY, Jiang GZ, Xie LX, Jiang ZG, Zhao G (2012) Air-fuel ratio intelligent control in coke oven combustion process. Information-An International Interdisciplinary Journal 15(11):4487–4494Google Scholar
  27. 27.
    Miao W, Li GF, Sun Y, Jiang GZ, Kong JY (2016) Gesture recognition based on sparse representation. Int J Wireless and Mobile Computing 11(4):348–356CrossRefGoogle Scholar
  28. 28.
    Li B, Sun Y, Li GF, Kong JY, Jiang GZ, Jiang D, Tao B, Xu S, Liu HH (2017) Gesture recognition based on modified adaptive orthogonal matching pursuit algorithm. Clust Comput.
  29. 29.
    He Y, Li GF, Liao YJ, Sun Y, Kong JY, Jiang GZ, Jiang D, Tao B, Xu S, Liu HH (2017) Gesture recognition based on an improved local sparse representation classification algorithm. Clust Comput.
  30. 30.
    Chen Z, Sun SK (2010) A zernike moment phase-based descriptor for local image representation and matching. IEEE Trans Image Process 19(1):205–219MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Chen DS, Li GF, Jiang GZ, Fang YF, Ju ZJ, Liu HH (2015) Intelligent computational control of multi-fingered dexterous robotic hand. Journal of Computational & Theoretical Nanoscience 12(12):6126–6132CrossRefGoogle Scholar
  32. 32.
    Ding WL, Li GF, Jiang GZ, Fang YF, Ju ZJ, Liu HH (2015) Intelligent computation in grasping control of dexterous robot hand. Journal of Computational & Theoretical Nanoscience 12(12):6096–6099CrossRefGoogle Scholar
  33. 33.
    Liu YQ, Wang XH, Yan K (2016). Hand gesture recognition based on concentric circular scan lines and weighted K-nearest neighbor algorithm. Multimedia Tools and Applications, pp1–15Google Scholar
  34. 34.
    Hugo JE, Eduardo FM, Enrique SL (2016) A naïve bayes baseline for early gesture recognition. Pattern Recogn Lett 73:91–99CrossRefGoogle Scholar
  35. 35.
    Kim J, Wagner J (2008) Bi-channel sensor fusion for automatic sign language recognition. 8th IEEE International Conference on Automatic Face & Gesture Recognition, pp647–652Google Scholar
  36. 36.
    Li GF, Liu Z, Jiang GZ, Xiong HG, Liu HH (2015) Numerical simulation of the influence factors for rotary kiln in temperature field and stress field and the structure optimization [J]. Advances in Mechanical Engineering 7(6):1–15CrossRefGoogle Scholar
  37. 37.
    Li GF, Qu PX, Kong JY, Jiang GZ, Xie LX, Wu ZH, Gao P, He Y (2013) Influence of working lining parameters on temperature and stress field of ladle. Applied Mathematics & Information Sciences 7(2):439–448CrossRefGoogle Scholar
  38. 38.
    Vidal C, Jedynak B (2010) Learning to match: deriving optimal template-matching algorithms from probabilistic image models. Int J Comput Vis 88(2):189–213MathSciNetCrossRefGoogle Scholar
  39. 39.
    Guo WC, Sheng XJ, Liu HG, Zhu XY (2017) Toward an enhanced human-machine interface for upper-limb prosthesis control with combined EMG and NIRS signals. IEEE Transactions on Human-machine Systems 47(4):564–575CrossRefGoogle Scholar
  40. 40.
    Rebecca W, Hussam M, Haydn M, Jiang XQ (2016) Burg algorithm for enhancing measurement performance in wavelength scanning interferometry. Surface Topography: Metrology and Properties 4:1–8Google Scholar
  41. 41.
    Ding WL, Li GF, Sun Y, Jiang GZ, Kong JY (2017) D-S evidential theory on sEMG signal recognition. Int J Comput Sci Math 8(2):138–145MathSciNetCrossRefGoogle Scholar
  42. 42.
    Liu S, Fu W, He L et al (2017) Distribution of primary additional errors in fractal encoding method [J]. Multimedia Tools and Applications 76(4):5787–5802CrossRefGoogle Scholar
  43. 43.
    Li Z, Li GF, Jiang GZ, Fang YF, Ju ZJ, Liu HH (2015) Intelligent Computation of grasping and manipulation for multi-fingered robotic hands. Journal of Computational & Theoretical Nanoscience 12(12):6192–6197CrossRefGoogle Scholar
  44. 44.
    Huang Z, Ren FJ (2017) Facial expression recognition based on multi-regional D-S evidences theory fusion. IEEJ Trans Electr Electron Eng 12(2):251–261CrossRefGoogle Scholar
  45. 45.
    Pan Z, Liu S (2017) A review of visual moving target tracking [J]. Multimedia Tools and Applications 76(16):16989–17018CrossRefGoogle Scholar
  46. 46.
    Liu S, Pan Z, Fu W, Cheng X (2017) Fractal generation method based on asymptote family of generalized Mandelbrot set and its application [J]. Journal of Nonlinear Sciences and Applications 10(3):1148–1161MathSciNetCrossRefGoogle Scholar
  47. 47.
    Li GF, Tang H, Sun Y, Kong JY, Jiang GZ, Jiang D, Tao B, Xu S, Liu HH (2017) Hand gesture recognition based on convolution neural network. Clust Comput.

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Key Laboratory of Metallurgical Equipment and Control Technology of Ministry of EducationWuhan University of Science and TechnologyWu HanChina
  2. 2.Hubei Key Laboratory of Mechanical Transmission and Manufacturing EngineeringWuhan University of Science and TechnologyWuhanChina
  3. 3.School of ComputingUniversity of PortsmouthPortsmouthUK
  4. 4.School of Computer Science and TechnologyHuazhong University of Science and TechnologyWuhanChina
  5. 5.College of Computer ScienceSouth-Central University for NationalitiesWuhanChina

Personalised recommendations