Abstract
In this paper we present an approach toward human action detection for activities of daily living (ADLs) that uses a convolutional neural network (CNN). The network is trained on discrete Fourier transform (DFT) images that result from raw sensor readings, i.e., each human action is ultimately described by an image. More specifically, we work using 3D skeletal positions of human joints, which originate from processing of raw RGB sequences enhanced by depth information. The motion of each joint may be described by a combination of three 1D signals, representing its coefficients into a 3D Euclidean space. All such signals from a set of human joints are concatenated to form an image, which is then transformed by DFT and is used for training and evaluation of a CNN. We evaluate our approach using a publicly available challenging dataset of human actions that may involve one or more body parts simultaneously and for two sets of actions which resemble common ADLs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abadi M et al (2016) TensorFlow: a system for large-scale machine learning. In: Proceedings of the USENIX symposium on operating systems design and implementation (OSDI)
Berretti S, Daoudi M, Turaga P, Basu A (2018) Representation, analysis, and recognition of 3D humans: a survey. ACM Trans Multim Comput Commun Appl (TOMM) 14(1s):16
Chollet F (2015) Keras. https://github.com/fchollet/keras
Du Y, Fu Y, Wang L (2015) Skeleton based action recognition with convolutional neural network. In: Proceedings of 3rd IAPR Asian conference on pattern recognition (ACPR). IEEE
Graves A, Mohamed AR, Hinton G (2013) Speech recognition with deep recurrent neural networks. In: Proceedings of IEEE international conference of acoustics, speech and signal processing (ICASSP)
Hou Y, Li Z, Wang P, Li W (2018) Skeleton optical spectra-based action recognition using convolutional neural networks. IEEE Trans Circuits Syst Video Technol 28(3):807–811
Jiang W, Yin Z (2015) Human activity recognition using wearable sensors by deep convolutional neural networks. In: Proceedings of ACM international conference on multimedia (MM)
Ke Q, An S, Bennamoun M, Sohel F, Boussaid F (2017) Skeletonnet: mining deep part features for 3-d action recognition. IEEE Signal Process Lett 24(6):731–735
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105
Lawton MP, Brody EM (1969) Assessment of older people: self-maintaining and instrumental activities of daily living. Gerontologist 9(3 Part 1):179–186
LeCun Y, Bottou L, Bengio Y, Haner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324
Li C, Hou Y, Wang P, Li W (2017) Joint distance maps based action recognition with convolutional neural networks. IEEE Signal Process Lett 24(5):624–628
Liu C, Hu Y, Li Y, Song S, Liu J (2017a) PKU-MMD: a large scale benchmark for continuous multi-modal human action understanding. In: Proceedings of ACM multimedia workshop (MM)
Liu M, Liu H, Chen C (2017b) Enhanced skeleton visualization for view invariant human action recognition. Pattern Recogn 68:346–362
Mathe E, Mitsou A, Spyrou E, Mylonas P (2018) Arm gesture recognition using a convolutional neural network. In: Proceedings of international workshop on semantic and social media adaptation and personalization (SMAP)
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958
Wang P, Li W, Ogunbona P, Wan J, Escalera S (2018a) RGB-D-based human motion recognition with deep learning: a survey. Comput Vis Image Underst 171:118–139
Wang P, Li W, Li C, Hou Y (2018b) Action recognition based on joint trajectory maps with convolutional neural networks. Knowl-Based Syst 158:43–53
Zhang Z (2012) Microsoft Kinect sensor and its effect. IEEE Multim 19(2):4–10
Acknowledgments
We acknowledge support for this work by the project SYNTELESIS “Innovative Technologies and Applications based on the Internet of Things (IoT) and the Cloud Computing” (MIS 5002521) which is implemented under the “Action for the Strategic Development on the Research and Technological Sector,” funded by the “Operational Programme Competitiveness, Entrepreneurship and Innovation” (NSRF 2014–2020) and co-financed by Greece and the European Union (European Regional Development Fund).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Mathe, E., Maniatis, A., Spyrou, E., Mylonas, P. (2020). A Deep Learning Approach for Human Action Recognition Using Skeletal Information. In: Vlamos, P. (eds) GeNeDis 2018. Advances in Experimental Medicine and Biology, vol 1194. Springer, Cham. https://doi.org/10.1007/978-3-030-32622-7_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-32622-7_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32621-0
Online ISBN: 978-3-030-32622-7
eBook Packages: Biomedical and Life SciencesBiomedical and Life Sciences (R0)