Abstract
A new method called Matrix Descriptor of Changes (MDC) is introduced in this work for description and recognition of human activity from sequences of skeletons. The primary focus was on one of the main problems in this area which is different duration of activities; it is assumed that the beginning and the end are known. Some existing methods use bag of features, hidden Markov models, recurrent neural networks or straighten the time interval by different sampling so that each activity has the same number of frames to solve this problem. The essence of our method is creating one or more matrices with a constant size. The sizes of matrices depend on the vector dimension containing the per-frame low-level features from which the matrix is created. The matrices then characterize the activity, even if we assume that certain activities may have different durations. The principle of this method is tested with two types of input features: (i) 3D position of the skeleton joints and (ii) invariant angular features of the skeleton. All kinds of feature types are processed by MDC separately and, in the subsequent step, all the information gathered together as a feature vector are used for recognition by Support Vector Machine classifier. Experiments have shown that the results are similar to results of the state-of-the-art methods. The primary contribution of proposed method was creating a new simple descriptor for activity recognition with preservation of the state-of-the-art results. This method also has a potential for parallel implementation and execution.
Keywords
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chaaraoui, A.A., Flrez-Revuelta, F.: Human action recognition optimization based on evolutionary feature subset selection. In: Proceeding of the Fifteenth Annual Conference on Genetic and Evolutionary Computation Conference - GECCO 2013 (2013)
Chen, C., Liu, K., Kehtarnavaz, N.: Real-time human action recognition based on depth motion maps. J. R.-Time Image Process. 12(1), 155–163 (2016)
Cippitelli, E., Gasparrini, S., Gambi, E., Spinsante, S.: A human activity recognition system using skeleton data from RGBD sensors. Comput. Intell. Neurosci. 2016, 114 (2016)
Cooper, M., Foote, J.: Scene boundary detection via video self-similarity analysis. In: Proceedings 2001 International Conference on Image Processing (Cat. No. 01CH37205), pp. 378–381. IEEE (2001)
Du, Y., Wang, W., Wang, L.: Hierarchical recurrent neural network for skeleton based action recognition. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)
Evangelidis, G., Singh, G., Horaud, R.: Skeletal quads: human action recognition using joint quadruples. In: 2014 22nd International Conference on Pattern Recognition (2014)
Foote, J.: Visualizing music and audio using self-similarity. In: Proceedings of the Seventh ACM International Conference on Multimedia (Part 1) - MULTIMEDIA 1999, pp. 77–80. ACM Press, New York (1999)
Gowayyed, M.A., Torki, M., Hussein, M.E., El-Saban, M.: Histogram of oriented displacements (HOD): describing trajectories of human joints for action recognition. In: Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, IJCAI 2013, pp. 1351–1357. AAAI Press (2013). http://dl.acm.org/citation.cfm?id=2540128.2540323
Junejo, I.N., Dexter, E., Laptev, I., Perez, P.: View-independent action recognition from temporal self-similarities. IEEE Trans. Pattern Anal. Mach. Intell. 33(1), 172–185 (2011)
Lee, I., Kim, D., Kang, S., Lee, S.: Ensemble deep learning for skeleton-based action recognition using temporal sliding LSTM networks. In: 2017 IEEE International Conference on Computer Vision (ICCV) (2017)
Li, W., Zhang, Z., Liu, Z.: Action recognition based on a bag of 3D points. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, pp. 9–14. IEEE (2010)
Liu, J., Shahroudy, A., Xu, D., Wang, G.: Spatio-temporal LSTM with trust gates for 3D human action recognition. CoRR abs/1607.07043 (2016). http://arxiv.org/abs/1607.07043
Liu, Z., Zhang, C., Tian, Y.: 3D-based deep convolutional neural network for action recognition with depth sequences. Image Vis. Comput. 55, 93100 (2016)
Ohn-Bar, E., Trivedi, M.M.: Joint angles similarities and HOG2 for action recognition. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 465–470. IEEE (2013)
Padilla-López, J.R., Chaaraoui, A.A., Flórez-Revuelta, F.: A discussion on the validation tests employed to compare human action recognition methods using the MSR Action3D dataset. CoRR abs/1407.7390 (2014). http://arxiv.org/abs/1407.7390
Seidenari, L., Varano, V., Berretti, S., Bimbo, A.D., Pala, P.: Recognizing actions from depth cameras as weakly aligned multi-part bag-of-poses. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops (2013)
Vemulapalli, R., Arrate, F., Chellappa, R.: Human action recognition by representing 3D skeletons as points in a lie group. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (2014)
Vieira, A.W., Nascimento, E.R., Oliveira, G.L., Liu, Z., Campos, M.F.: On the improvement of human action recognition from depth map sequences using spacetime occupancy patterns. Pattern Recogn. Lett. 36, 221–227 (2014)
Wang, J., Liu, Z., Chorowski, J., Chen, Z., Wu, Y.: Robust 3D action recognition with random occupancy patterns. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, pp. 872–885. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33709-3_62
Wang, J., Liu, Z., Wu, Y., Yuan, J.: Mining actionlet ensemble for action recognition with depth cameras. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1290–1297. IEEE (2012)
Wang, P., Li, W., Ogunbona, P., Wan, J., Escalera, S.: RGB-D-based human motion recognition with deep learning: a survey. arXiv e-prints, October 2017
Xia, L., Chen, C.C., Aggarwal, J.K.: View invariant human action recognition using histograms of 3D joints. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 20–27. IEEE (2012)
Yang, R., Yang, R.: DMM-pyramid based deep architectures for action recognition with depth cameras. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9007, pp. 37–49. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16814-2_3
Yang, X., Tian, Y.: Effective 3D action recognition using eigenjoints. J. Vis. Commun. Image Represent. 25(1), 211 (2014)
Yang, X., Zhang, C., Tian, Y.: Recognizing actions using depth motion maps-based histograms of oriented gradients. In: Proceedings of the 20th ACM International Conference on Multimedia - MM 2012 (2012)
Zhang, S., Liu, X., Xiao, J.: On geometric features for skeleton-based action recognition using multilayer LSTM networks. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) (2017)
Zhu, Y., Chen, W., Guo, G.: Fusing spatiotemporal features and joints for 3D action recognition. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops (2013)
Acknowledgements
This work was partially supported by Grant of SGS No. SP2018/42, VŠB - Technical University of Ostrava, Czech Republic.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Simkanič, R. (2018). Matrix Descriptor of Changes (MDC): Activity Recognition Based on Skeleton. In: Blanc-Talon, J., Helbert, D., Philips, W., Popescu, D., Scheunders, P. (eds) Advanced Concepts for Intelligent Vision Systems. ACIVS 2018. Lecture Notes in Computer Science(), vol 11182. Springer, Cham. https://doi.org/10.1007/978-3-030-01449-0_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-01449-0_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-01448-3
Online ISBN: 978-3-030-01449-0
eBook Packages: Computer ScienceComputer Science (R0)