Abstract
First-person video summarization has emerged as an important problem in the areas of computer vision and multimedia communities. In this paper, we present a graph-theoretic framework for summarizing first-person (egocentric) videos at frame level. We first develop a new way of characterizing egocentric video frames by building a center-surround model based on spectral measures of dissimilarity between two graphs representing the center and the surrounding regions in a frame. The frames in a video are next represented by a weighted graph (video similarity graph) in the feature space constituting center-surround differences in entropy and optic flow values along with PHOG (Pyramidal HOG) features. The frames are finally clustered using a MST based approach with a new measure of inadmissibility for edges based on neighbourhood analysis. Frames closest to the centroid of each cluster are used to build the summary. Experimental comparisons on two standard datasets clearly indicate the advantage of our solution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Del Molino, A.G., Tan, C., Lim, J.H., Tan, A.H.: Summarization of egocentric videos: a comprehensive survey. IEEE THMS 47(1), 65–76 (2017)
Lee, Y.J., Ghosh, J., Grauman, K.: Discovering important people and objects for egocentric video summarization. In: CVPR, pp. 1346–1353 (2012)
Lu, Z., Grauman, K.: Story-driven summarization for egocentric video. In: CVPR, pp. 2714–2721 (2013)
Gygli, M., Grabner, H., Riemenschneider, H., Van Gool, L.: Creating summaries from user videos. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8695, pp. 505–520. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10584-0_33
Song, Y., Vallmitjana, J., Stent, A., Jaimes, A.: TVSum: summarizing web videos using titles. In: CVPR, pp. 5179–5187 (2015)
Gygli, M., Grabner, H., Van Gool, L.: Video summarization by learning submodular mixtures of objectives. In: CVPR, pp. 3090–3098 (2015)
Zhang, K., Chao, W.-L., Sha, F., Grauman, K.: Video summarization with long short-term memory. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 766–782. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46478-7_47
Otani, M., Nakashima, Y., Rahtu, E., Heikkilä, J., Yokoya, N.: Video summarization using deep semantic features. In: Lai, S.-H., Lepetit, V., Nishino, K., Sato, Y. (eds.) ACCV 2016. LNCS, vol. 10115, pp. 361–377. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54193-8_23
Mahasseni, B., Lam, M., Todorovic, S.: Unsupervised video summarization with adversarial LSTM networks. In: CVPR, pp. 2982–2991 (2017)
Guo, Z., Gao, L., Zhen, X., Zou, F., Shen, F., Zheng, K.: Spatial and temporal scoring for egocentric video summarization. Neurocomputing 208, 299–308 (2016)
Arbelaitz, O., Gurrutxaga, I., Muguerza, J., Pérez, J.M., Perona, I.: An extensive comparative study of cluster validity indices. Pattern Recogn. 46(1), 243–256 (2013)
Almeida, J., Leite, N.J., Torres, R.D.S.: Vison: video summarization for online applications. Pattern Recogn. Lett. 33(4), 97–409 (2012)
Guimarães, S.J.F., Gomes, W.: A static video summarization method based on hierarchical clustering. In: Bloch, I., Cesar, R.M. (eds.) CIARP 2010. LNCS, vol. 6419, pp. 46–54. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-16687-7_11
Dos Santos Belo, L., Caetano Jr., C.A., do PatrocÃnio Jr., Z.K.G., Guimarães, S.J.F.: Summarizing video sequence using a graph-based hierarchical approach. Neurocomputing 173, 1001–1016 (2016)
Sahu, A., Chowdhury, A.S.: Shot level egocentric video co-summarization. In: ICPR, pp. 2887–2892 (2018)
Gangapure, V.N., Nanda, S., Chowdhury, A.S., Jiang, X.: Causal video segmentation using superseeds and graph matching. In: Liu, C.-L., Luo, B., Kropatsch, W.G., Cheng, J. (eds.) GbRPR 2015. LNCS, vol. 9069, pp. 282–291. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-18224-7_28
Bosch, A., Zisserman, A., Munoz, X.: Representing shape with a spatial pyramid kernel. In: CIVR, pp. 401–408 (2007)
Zahn, C.T.: Graph theoretical methods for detecting and describing gestalt clusters. IEEE Trans. Comput. C–20, 68–86 (1970)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Sahu, A., Chowdhury, A.S. (2019). A Graph-Theoretic Framework for Summarizing First-Person Videos. In: Conte, D., Ramel, JY., Foggia, P. (eds) Graph-Based Representations in Pattern Recognition. GbRPR 2019. Lecture Notes in Computer Science(), vol 11510. Springer, Cham. https://doi.org/10.1007/978-3-030-20081-7_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-20081-7_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-20080-0
Online ISBN: 978-3-030-20081-7
eBook Packages: Computer ScienceComputer Science (R0)
