Abstract
Precisely describing the action inside of a video is a challenging task because the content of the video includes various objects, with different local motion information at different speed in the video frames. In this paper, a new video feature is proposed based on the spatial information of the objects in a frame, along with the motion information between one against multiple consecutive frames. Motion information between pixels at the same position in the whole video are all combined for a new Spatial Multi-Scale Motion History Histogram (SMMHH) dynamic descriptor. The detailed algorithm of the SMMHH was given and it is tested in both human action recognition and touch gesture recognition applications based on the public video datasets. Experimental results demonstrate its excellent performance compared to other traditional methods.
A. Jan and Z. Zhao—Equal contribution.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Meng, H., Pears, N., Freeman, M., Bailey, C.: Motion history histograms for human action recognition. In: Embedded Computer Vision, pp. 139–162. Springer (2009)
Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 6, 915–928 (2007)
Davis, J.W., Bobick, A.F.: The representation and recognition of action using temporal templates. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 928–934 (1997)
Weinland, D., Ronfard, R., Boyer, E.: Free viewpoint action recognition using motion history volumes. Comput. Vis. Image Underst. 104(2–3), 249–257 (2006)
Ojala, T., Pietikäinen, M., Mäenpää, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 7, 971–987 (2002)
Mattivi, R., Shao, L.: Human action recognition using LBP-top as sparse spatio-temporal feature descriptor. In: International Conference on Computer Analysis of Images and Patterns, pp. 740–747. Springer (2009)
Almaev, T.R., Valstar, M.F.: Local gabor binary patterns from three orthogonal planes for automatic facial expression recognition. In: 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, pp. 356–361. IEEE (2013)
Senechal, T., Rapp, V., Salam, H., Seguier, R., Bailly, K., Prevost, L.: Facial action recognition combining heterogeneous features via multikernel learning. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 42(4), 993–1005 (2012)
Jiang, B., Valstar, M.F., Pantic, M.: Action unit detection using sparse appearance descriptors in space-time video volumes. In: International Conference on Face and Gesture Recognition, pp. 314–321. IEEE (2011)
Klaser, A., Marszałek, M., Schmid, C.: A spatio-temporal descriptor based on 3d-gradients. In: BMVC 2008-19th British Machine Vision Conference, vol. 275, pp. 1–10. British Machine Vision Association (2008)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: International Conference on computer vision and Pattern Recognition (CVPR 2005), vol. 1, pp. 886–893. IEEE Computer Society (2005)
Meng, H., Pears, N., Bailey, C.: A human action recognition system for embedded computer vision application. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–6. IEEE (2007)
Meng, H., Pears, N.: Descriptive temporal template features for visual motion recognition. Pattern Recogn. Lett. 30(12), 1049–1058 (2009)
Schuldt, C., Laptev, I., Caputo, B.: Recognizing human actions: a local SVM approach. In: Proceedings of the 17th International Conference on Pattern Recognition, pp. 32–36 (2004)
Laptev, I., Marszałek, M., Schmid, C., Rozenfeld, B.: Learning realistic human actions from movies. In:CVPR 2008-IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE Computer Society (2008)
Jung, M.M., Poppe, R., Poel, M., Heylen, D.K.J.: Touching the void–introducing cost: corpus of social touch. In: Proceedings of the 16th International Conference on Multimodal Interaction, pp. 120–127. ACM (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Jan, A., Zhao, Z., Chen, T., Meng, H., Lei, T. (2020). Spatial Multi-scale Motion History Histograms and Its Applications. In: Liu, Y., Wang, L., Zhao, L., Yu, Z. (eds) Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery. ICNC-FSKD 2019. Advances in Intelligent Systems and Computing, vol 1075. Springer, Cham. https://doi.org/10.1007/978-3-030-32591-6_79
Download citation
DOI: https://doi.org/10.1007/978-3-030-32591-6_79
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32590-9
Online ISBN: 978-3-030-32591-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)