A Generic Multi-modal Dynamic Gesture Recognition System Using Machine Learning
Human computer interaction facilitates intelligent communication between humans and computers, in which gesture recognition plays a prominent role. This paper proposes a machine learning system to identify dynamic gestures using tri-axial acceleration data acquired from two public datasets. These datasets, uWave and Sony, were acquired using accelerometers embedded in Wii remotes and smartwatches, respectively. A dynamic gesture signed by the user is characterized by a generic set of features extracted across time and frequency domains. The system was analyzed from an end-user perspective and was modelled to operate in three modes. The modes of operation determine the subsets of data to be used for training and testing the system. From an initial set of seven classifiers, three were chosen to evaluate each dataset across all modes rendering the system towards mode-neutrality and dataset-independence. The proposed system is able to classify gestures performed at varying speeds with minimum preprocessing, making it computationally efficient. Moreover, this system was found to run on a low-cost embedded platform – Raspberry Pi Zero (USD 5), making it economically viable.
KeywordsGesture recognition Accelerometers Feature extraction Machine learning algorithms
The authors would like to thank Solarillion Foundation for its support and funding of the research work carried out.
- 1.Chen, Q., Georganas, N.D., Petriu, E.M.: Real-time vision-based hand gesture recognition using HAAR-like features. In: 2007 IEEE Instrumentation & Measurement Technology Conference (IMTC 2007), Warsaw, pp. 1–6 (2007)Google Scholar
- 2.Biswas, K.K., Basu, S.K.: Gesture recognition using Microsoft Kinect. In: The 5th International Conference on Automation, Robotics and Applications, Wellington, pp. 100–103 (2011)Google Scholar
- 4.Liu, J., Wang, Z., Zhong, L., Wickramasuriya, J., Vasudevan, V.: uWave: accelerometer-based personalized gesture recognition and its applications. In: 2009 IEEE International Conference on Pervasive Computing and Communications, Galveston, TX, pp. 1–9 (2009)Google Scholar
- 5.SmartWatch Gestures Dataset, Technologies of Vision, Fondazione Bruno Kessler. https://tev.fbk.eu/technologies/smartwatch-gestures-dataset
- 6.Zimmerman, T.G., Lanier, J., Blanchard, C., Bryson, S., Harvill, Y.: Hand gesture interface device. In: Proceedings of the SIGCHI/GI Conference on Human Factors in Computing Systems and Graphics Interface, pp. 189–192 (1986)Google Scholar
- 7.Hussain, S.M.A., Rashid, A.B.M.H.: User independent hand gesture recognition by accelerated DTW. In: 2012 International Conference on Informatics, Electronics & Vision (ICIEV), Dhaka, pp. 1033–1037 (2012)Google Scholar
- 8.Pylvänäinen, T.: Accelerometer based gesture recognition using continuous HMMs. In: IbPRIA, vol. 3522, pp. 639–646 (2005). 492Google Scholar
- 9.Helmi, N., Helmi, M.: Applying a neuro-fuzzy classifier for gesture-based control using a single wrist-mounted accelerometer. In: IEEE International Symposium on Computational Intelligence in Robotics and Automation - (CIRA), Daejeon, pp. 216–221 (2009)Google Scholar
- 14.Tencer, L., Reznáková, M., Cheriet, M.: Evaluation of techniques for signature classification from accelerometer and gyroscope data. In: ICDAR, pp. 1066–1070 (2015)Google Scholar
- 15.Aswolinskiy, W., Reinhart, R.F., Steil, J.J.: Impact of regularization on the model space for time series classification. In: Machine Learning Reports, pp. 49–56 (2015)Google Scholar
- 16.Yang, Y., Yu, Y.: A hand gestures recognition approach combined attribute bagging with symmetrical uncertainty. In: 2012 9th International Conference on Fuzzy Systems and Knowledge Discovery, Sichuan, pp. 2551–2554 (2012)Google Scholar