Abstract
The task of long video analysis is challenging, and it is often the case that many human actions occur but only a few contribute to the semantic topic of the video. However, compared with short video human activity studies, long video analysis has its practical utility especially considering the effort of watching a long video for human. In this paper, we propose to learn semantic symbol sequence patterns of complex videos for activity prediction. The prefix method of semantic stream is designed based on the semantic symbol sequence and their time marks. The prediction phase is implemented via matching semantic sequence of incomplete videos and sequence patterns of different activities. We evaluate various prediction methods depending on low-level features or high-level descriptions. The empirical result suggests that when applied to activity prediction, sequence pattern mining can effectively reduce its reliance upon the low level features and improve predicting performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Yuan, C., Li, X., Hu, W.M., et al.: 3D R transform on spatio-temporal interest points for action recognition. In: CVPR, pp. 724–730 (2013)
Soomro, K., Idrees, H., Shah, M.: Predicting the where and what of actors and actions through online action localization. In: CVPR, pp. 2648–2657 (2016)
Wang, L., Zhao, X., Cao, L., et al.: Context-associative hierarchical memory model for human activity recognition and prediction. IEEE Trans. Multimedia 19(3), 646–659 (2017)
Seo, H.J., Milanfar, P.: Action recognition from one example. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 867–882 (2011)
Yan, J., Zhu, M., Liu, H., et al.: Visual saliency detection via sparsity pursuit. IEEE Signal Process. Lett. 17(8), 739–742 (2010)
Gan, C., et al.: DevNet: a deep event network for multimedia event detection and evidence recounting. In: CVPR, pp. 2568–2577 (2015)
Liu, W,. Mei, T., Zhang, Y., et al.: Multi-task deep visual-semantic embedding for video thumbnail selection. In: CVPR, pp. 3707–3715 (2015)
Kitani, K.M., Okabe, T., Sato, Y., et al.: Fast unsupervised ego-action learning for first-person sports videos. In: CVPR, pp. 3241–3248 (2011)
Ryoo, M.S., Aggarwal, J.K.: Observe-and-explain: a new approach for multiple hypotheses tracking of humans and objects. In: CVPR, pp. 1–8 (2008)
Li, Y., Zhou, Y., Yan, J., Niu, Z., Yang, J.: Visual saliency based on conditional entropy. In: Zha, H., Taniguchi, R., Maybank, S. (eds.) ACCV 2009. LNCS, vol. 5994, pp. 246–257. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-12307-8_23
Ryoo, M.S.: Human activity prediction: early recognition of ongoing activities from streaming videos. In: Proceedings, vol. 24(4), pp. 1036–1043 (2011)
Kitani, K.M., Ziebart, B.D., Bagnell, J.A., Hebert, M.: Activity forecasting. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7575, pp. 201–214. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33765-9_15
Li, K., Fu, Y.: Prediction of human activity by discovering temporal sequence patterns. IEEE Trans. Pattern Anal. Mach. Intell. 36(8), 1644–1657 (2014)
Cheng, Y., Fan, Q., Pankanti, S., et al.: Temporal sequence modeling for video event detection. In: CVPR, pp. 2235–2242 (2014)
Xu, Z., Qing, L., Miao, J.: Activity auto-completion: predicting human activities from partial videos. In: ICCV, pp. 3191–3199 (2015)
Hu, J.-F., Zheng, W.-S., Ma, L., Wang, G., Lai, J.: Real-time RGB-D activity prediction by soft regression. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 280–296. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_17
Guyet, T., Quiniou, R.: Extracting temporal patterns from interval-based sequences. In: IJCAI, pp. 1306–1311 (2014)
Aggarwal, C.C., Han, J.W.: Frequent Pattern Mining. Springer, New York (2014). https://doi.org/10.1007/978-3-319-07821-2
Laptev, I., Perez, P.: Retrieving actions in movies. In: ICCV, pp. 1–8 (2007)
Schiele, B., Andriluka, M., Amin, S., et al.: A database for fine grained activity detection of cooking activities. In: CVPR, pp. 1194–1201 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Wang, G., Qin, Z., Xu, K. (2018). Semantic Sequence Analysis for Human Activity Prediction. In: Zeng, B., Huang, Q., El Saddik, A., Li, H., Jiang, S., Fan, X. (eds) Advances in Multimedia Information Processing – PCM 2017. PCM 2017. Lecture Notes in Computer Science(), vol 10735. Springer, Cham. https://doi.org/10.1007/978-3-319-77380-3_26
Download citation
DOI: https://doi.org/10.1007/978-3-319-77380-3_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-77379-7
Online ISBN: 978-3-319-77380-3
eBook Packages: Computer ScienceComputer Science (R0)