Research on gesture recognition of smart data fusion features in the IoT

  • Chong Tan
  • Ying Sun
  • Gongfa LiEmail author
  • Guozhang Jiang
  • Disi Chen
  • Honghai Liu
Smart Data Aggregation Inspired Paradigm & Approaches in IoT Applns


With the rapid development of Internet of things technology, the interaction between people and things has become increasingly frequent. Using simple gestures instead of complex operations to interact with the machine, the fusion of smart data feature information and so on has gradually become a research hotspot. Considering that the depth image of the Kinect sensor lacks color information and is susceptible to depth thresholds, this paper proposes a gesture segmentation method based on the fusion of color information and depth information; in order to ensure the complete information of the segmentation image, a gesture feature extraction method based on Hu invariant moment and HOG feature fusion is proposed; and by determining the optimal weight parameters, the global and local features are effectively fused. Finally, the SVM classifier is used to classify and identify gestures. The experimental results show that the proposed fusion features method has a higher gesture recognition rate and better robustness than the traditional method.


Gesture recognition Fusion features Smart data aggregation Hu moment SVM 



This work was supported by grants of the National Natural Science Foundation of China (Grant Nos. 51575407, 51575338, 51575412, 51505349 and 61273106) and Grants of the National Defense Pre-Research Foundation of Wuhan University of Science and Technology (GF201705).

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. 1.
    Chen D, Li G, Sun Y, Kong J, Jiang G, Tang H, Ju Z, Yu H, Liu H (2017) An interactive image segmentation method in hand gesture recognition. Sensors 17:253. CrossRefGoogle Scholar
  2. 2.
    Sun Y, Li C, Li G, Jiang G, Jiang D, Liu H, Zheng Z, Shu W (2018) Gesture recognition based on kinect and sEMG signal fusion. Mobile Netw Appl 23(4):797–805CrossRefGoogle Scholar
  3. 3.
    Li G, Zhang L, Sun Y, Kong J (2018) Towards the sEMG hand: internet of things sensors and haptic feedback application. Multimed Tools Appl. Google Scholar
  4. 4.
    He Y, Li G, Sun Y, Zhao Y, Jiang G (2018) Numerical simulation-based optimization of contact stress distribution and lubrication conditions in the straight worm drive. Strength Mater 50(11):157. CrossRefGoogle Scholar
  5. 5.
    Jadooki S, Mohamad D, Saba T, Almazyad A, Rehman A (2017) Fused features mining for depth-based hand gesture recognition to classify blind human communication. Neural Comput Appl 28:3285. CrossRefGoogle Scholar
  6. 6.
    Li G, Tang H, Sun Y, Kong J, Jiang G, Jiang D, Tao B, Xu S, Liu H (2017) Hand gesture recognition based on convolution neural network. Clust Comput. Google Scholar
  7. 7.
    Sun Y, Li G, Kong J, Jiang G, Jiang D, Tao B, Xu S, Liu H (2017) Gesture recognition based on modified adaptive orthogonal matching pursuit algorithm. Clust Comput. Google Scholar
  8. 8.
    Chang W, Li G, Kong J, Sun Y, Jiang G, Liu H (2018) Thermal mechanical stress analysis of ladle lining with integral brick joint. Arch Metall Mater 63(2):659–666Google Scholar
  9. 9.
    Liu H, Li G, Zhu X (2015) A multichannel surface EMG system for hand motion recognition. Int J Humanoid Robot. Google Scholar
  10. 10.
    Yin Q, Li G, Zhu J (2017) Research on the method of step feature extraction for EOD robot based on 2D laser radar. Discrete Contin Dyn Syst Ser S 8(6):1415–1421MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Chen D, Li G, Sun Y, Jiang G, Kong J, Li J, Liu H (2017) Fusion hand gesture segmentation and extraction based on CMOS sensor and 3D sensor. Int J Wirel Mobile Comput 12(3):305–312CrossRefGoogle Scholar
  12. 12.
    Sun Y, Hu J, Li G, Jiang G, Xiong H, Tao B, Zheng Z, Jiang D (2018) Gear reducer optimal design based on computer multimedia simulation. J Supercomput. Google Scholar
  13. 13.
    He L, Xiong C, Liu K, Huang J, He C, Chen W (2018) Mechatronic design of a synergetic upper limb exoskeletal robot and wrench-based assistive control. J Bionic Eng 15:247. CrossRefGoogle Scholar
  14. 14.
    Cui Y, Xiong C (2014) Dynamic recurrent neural network based classification scheme for myoelectric control of upper limb rehabilitation robot. J Mech Med Biol. Google Scholar
  15. 15.
    Jiang D, Li G, Sun Y, Kong J, Tao B (2018) Gesture recognition based on skeletonization algorithm and CNN with ASL database. Multimed Tools Appl. Google Scholar
  16. 16.
    Cheng W, Sun Y, Li G, Jiang G, Liu H (2018) Jointly network: a network based on CNN and RBM for gesture recognition. Neural Comput Appl. Google Scholar
  17. 17.
    Li J, Liu X, Ouyang G (2016) Using relevance feedback to distinguish the changes in EEG during different absence seizure phases. ClinicalEeg Neurosci. Google Scholar
  18. 18.
    Li G, Qu P, Kong J, Jiang G, Xie L, Gao P, Wu Z, He Y (2013) Coke oven intelligent integrated control system. Appl Math Inf Sci 7(3):1043–1050CrossRefGoogle Scholar
  19. 19.
    Patvardhan C, Kumar P, Lakshmi CV (2018) Effective color image watermarking scheme using YCbCr color space and QR code. Multimed Tools Appl 77(10):12655. CrossRefGoogle Scholar
  20. 20.
    Li G, Miao W, Jiang G, Fang Y, Ju Z, Liu H (2015) Intelligent control model and its simulation of flue temperature in coke oven. Discrete Contin Dyn Syst Ser S 8(6):1223–1237MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Kundu A, Mazumder O, Lenka P, Bhaumik S (2017) Hand gesture recognition based omnidirectional wheelchair control Using IMU and EMG sensors. J Intell Robot Syst 91:529. CrossRefGoogle Scholar
  22. 22.
    Li G, Gu Y, Kong J, Jiang G, Xie L, Wu Z, Li Z, He Y, Gao P (2013) Intelligent control of air compressor production process. Appl Math Inf Sci 7(3):1051–1058CrossRefGoogle Scholar
  23. 23.
    Fei M, Li J, Liu H (2015) Visual tracking based on improved foreground detection and perceptual hashing. Neurocomputing 152:413–428CrossRefGoogle Scholar
  24. 24.
    Li G, Jiang D, Zhou Y, Jiang G, Kong J, Manogaran G (2019) Human lesion detection method based on image information and brain signal. IEEE Access. Google Scholar
  25. 25.
    Li G, Kong J, Jiang G, Xie L, Jiang Z, Zhao G (2012) Air-fuel ratio intelligent control in coke oven combustion process. Int J Inf 12(11):4487–4494Google Scholar
  26. 26.
    He J, Zhu X (2017) Combining improved gray-level co-occurrence matrix with high density grid for myoelectric control robustness to electrode shift. IEEE Trans Neural Syst Rehabil Eng 99:1539–1548CrossRefGoogle Scholar
  27. 27.
    Zhu X, Liu J, Zhang D, Sheng X, Jiang N (2017) Cascaded adaptation framework for fast calibration of myoelectric control. IEEE Trans Neural Syst Rehabil Eng 25(3):254–264CrossRefGoogle Scholar
  28. 28.
    Hu C, Arvin F, Xiong C, Yue S (2017) Bio-inspired embedded vision system for autonomous micro-robots: the LGMD case. IEEE Trans Cognit Dev Syst 9(3):241–254CrossRefGoogle Scholar
  29. 29.
    Huang Y, Wang Y, Xiao L, Dong W (2014) Microfluidic serpentine antennas with designed mechanical tunability. Lab Chip 14(21):4205–4212CrossRefGoogle Scholar
  30. 30.
    Jiang D, Zheng Z, Li G, Sun Y, Kong J, Jiang G, Xiong H, Tao B, Xu S, Yu H, Liu H, Ju Z (2018) Gesture recognition based on binocular vision. Clust Comput. Google Scholar
  31. 31.
    He Y, Li G, Liao Y, Sun Y, Kong J, Jiang G, Jiang D, Tao B, Xu S, Liu H (2018) Gesture recognition based on an improved local sparse representation classification algorithm. Clust Comput. Google Scholar
  32. 32.
    Liao Y, Sun Y, Li G, Kong J, Jiang G, Jiang D, Cai H, Ju Z, Yu H, Liu H (2017) Simultaneous calibration: a joint optimization approach for multiple kinect and external cameras. Sensors 17(7):1491. CrossRefGoogle Scholar
  33. 33.
    Miao W, Li G, Sun Y, Jiang G, Kong J, Liu H (2016) Gesture recognition based on sparse representation. Int J Wirel Mobile Comput 11(4):348–356CrossRefGoogle Scholar
  34. 34.
    Li J, Wang J, Ju Z (2018) A novel hand gesture recognition based on high-level features. Int J Humanoid Rob. Google Scholar
  35. 35.
    Ju Z, Ji X, Li J, Liu H (2017) An integrative framework of human hand gesture segmentation for human-robot interaction. IEEE Syst J 11(3):1326–1336CrossRefGoogle Scholar
  36. 36.
    Li J, Allinson NM (2013) Building recognition using local oriented features. IEEE Trans Industr Inf 9(3):1697–1704CrossRefGoogle Scholar
  37. 37.
    Oyedotun OK, Khashman A (2017) Deep learning in vision-based static hand gesture recognition. Neural Comput Appl 28:3941. CrossRefGoogle Scholar
  38. 38.
    Miao W, Li G, Jiang G, Fang Y, Ju Z, Liu H (2015) Optimal grasp planning of multi-fingered robotic hands: a review. Appl Comput Math 14(3):238–247MathSciNetzbMATHGoogle Scholar
  39. 39.
    Liu H, Wu J, Fan S, Jin M, Fan C (2018) Integrated virtual impedance control based pose correction for a simultaneous three-fingered end-effector. Ind Robot. Google Scholar
  40. 40.
    Li Z, Wang B, Liu H (2016) Target capturing control for space robots with unknown mass properties: a self-tuning method based on gyros and cameras. Sensors 16(9):1383. CrossRefGoogle Scholar
  41. 41.
    Cui M, Prasad S (2015) Class-dependent sparse representation classifier for robust hyperspectral image classification. IEEE Trans Geosci Remote Sens 53(5):2683–2695CrossRefGoogle Scholar
  42. 42.
    Liu H, Yang D, Jiang L, Fan S (2014) Development of a multi-DOF prosthetic hand with intrinsic actuation, intuitive control and sensory feedback. Ind Robot Int J 41(4):381–392CrossRefGoogle Scholar
  43. 43.
    Shull PB, Zhu X, Cutkosky MR (2017) Continuous movement tracking performance for predictable and unpredictable tasks with vibrotactile feedback. IEEE Trans Haptics 10(4):466–475CrossRefGoogle Scholar
  44. 44.
    Huang Y, Dong W, Huang T, Wang Y, Xiao L (2015) Self-similar design for stretchable wireless LC strain sensors. Sens Actuators A. Google Scholar
  45. 45.
    Li G, Wu H, Jiang G, Xu S, Liu H (2018) Dynamic gesture recognition in the internet of things. IEEE Access. Google Scholar
  46. 46.
    Bellocerezo R, Bianconi F, Fernández A, González E, Maria FD (2016) Experimental comparison of color spaces for material classification. J Electron Imaging. Google Scholar
  47. 47.
    Chai G, Zhang D, Zhu X (2017) Developing non-somatotopic phantom finger sensation to comparable levels of somatotopic sensation through user training with electrotactile stimulation. IEEE Trans Neural Syst Rehabil Eng 25(5):469–480CrossRefGoogle Scholar
  48. 48.
    Li C, Li G, Jiang G, Chen D, Liu H (2018) Surface EMG data aggregation processing for intelligent prosthetic action recognition. Neural Comput Appl. Google Scholar
  49. 49.
    Satapathy S, Sri Madhava Raja N, Rajinikanth V, Ashour A (2018) Multi-level image thresholding using Otsu and chaotic bat algorithm. Neural Comput Appl 29:1285. CrossRefGoogle Scholar
  50. 50.
    Dong W, Gu G, Zhu X, Dong X (2015) Solving the boundary value problem of an under-actuated quadrotor with subspace stabilization approach. J Intell Rob Syst 80:299. CrossRefGoogle Scholar
  51. 51.
    Wu H, Huang Y, Xu F, Xu F, Duan Y, Yin Z (2016) Energy harvesters for wearable and stretchable electronics: from flexibility to stretchability. Adv Mater. Google Scholar
  52. 52.
    Radman A, Zainal N, Suandi SA (2017) Automated segmentation of iris images acquired in an unconstrained environment using HOG-SVM and GrowCut. Digit Signal Proc 64:60–70MathSciNetCrossRefGoogle Scholar
  53. 53.
    Jebril NA, Al-Zoubi HR, Al-Haija QA (2018) Recognition of handwritten arabic characters using histograms of oriented gradient (HOG). Pattern Recognit Image Anal 28:321. CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  • Chong Tan
    • 1
  • Ying Sun
    • 1
    • 2
  • Gongfa Li
    • 1
    • 2
    • 4
    Email author
  • Guozhang Jiang
    • 2
    • 3
  • Disi Chen
    • 5
  • Honghai Liu
    • 5
  1. 1.Key Laboratory of Metallurgical Equipment and Control Technology of Ministry of EducationWuhan University of Science and TechnologyWuhanChina
  2. 2.Hubei Key Laboratory of Mechanical Transmission and Manufacturing EngineeringWuhan University of Science and TechnologyWuhanChina
  3. 3.Research Center of Biologic Manipulator and Intelligent Measurement and ControlWuhan University of Science and TechnologyWuhanChina
  4. 4.Institute of Precision ManufacturingWuhan University of Science and TechnologyWuhanChina
  5. 5.School of ComputingUniversity of PortsmouthPortsmouthUK

Personalised recommendations