Abstract
Automatic concept detection is a crucial aspect of automatically indexing unstructured multimedia archives. However, the current prevalence of one-per-class detectors neglect inherent concept relationships and operate in isolation. This is insufficient when analyzing content gathered from wearable visual sensing, in which concepts occur with high diversity and with correlation depending on context. This paper presents a method to enhance concept detection results by constructing and factorizing a multi-way concept detection tensor in a time-aware manner. We derived a weighted non-negative tensor factorization algorithm and applied it to model concepts’ temporal occurrence patterns and show how it boosts overall detection performance. The potential of our method is demonstrated on lifelog datasets with varying levels of original concept detection accuracies.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Gurrin, C., Smeaton, A.F., Doherty, A.: LifeLogging: personal big data. Foundations and Trends in Information Retrieval 8(1), 1–127 (2014)
Smeaton, A., Over, P., Kraaij, W.: High level feature detection from video in TRECVid: a 5-year retrospective of achievements. In: Divakaran, A. (ed.) Multimedia Content Analysis, Theory and Applications, pp. 151–174. Springer (2008)
Doherty, A.R., Pauly-Takacs, K., Caprani, N., Gurrin, C., Moulin, C.J.A., O’Connor, N.E., Smeaton, A.F.: Experiences of aiding autobiographical memory using the SenseCam. Human-Computer Interaction 27(1-2), 151–174 (2012)
Doherty, A.R., Smeaton, A.F.: Automatically segmenting lifelog data into events. In: WIAMIS 2008, pp. 20–23. IEEE Computer Society, Washington, DC (2008)
Byrne, D., Doherty, A.R., Snoek, C.G.M., Jones, G.J.F., Smeaton, A.F.: Everyday concept detection in visual lifelogs: validation, relationships and trends. Multimedia Tools Appl. 49(1), 119–144 (2010)
Doherty, A.R., Caprani, N., O’Conaire, C., Kalnikaite, V., Gurrin, C., O’Connor, N.E., Smeaton, A.F.: Passively recognising human activities through lifelogging. Computers in Human Behavior 27(5), 1948–1958 (2011)
Aly, R., Hiemstra, D., de Jong, F., Apers, P.: Simulating the future of concept-based video retrieval under improved detector performance. Multimedia Tools and Applications 60(1), 1–29 (2011)
Tamara, G., Kolda, B.W.: Bader: Tensor decompositions and applications. SIAM Review 51(3), 455–500 (2009)
Tamara, G.: Kolda: Multilinear operators for higher-order decompositions. Tech. Report SAND2006-2081 (2006)
Rendle, S., Schmidt-Thieme, L.: pairwise interaction tensor factorization for personalized tag recommendation. In: WSDM 2010, pp. 81–90 (2010)
Qi, G.J., Hua, X.S., Rui, Y., Tang, J., Mei, T., Zhang, H.J.: Correlative multi-label video annotation. In: ACM MM 2007, pp. 17–26 (2007)
Wu, Y., Tseng, B., Smith, J.: Ontology-based multi-classification learning for video concept detection. In: ICME 2004, vol. 2, pp. 1003–1006 (2004)
Wang, P., Smeaton, A.F.: Using visual lifelogs to automatically characterise everyday activities. Information Sciences 230, 147–161 (2013)
Smith, J.R., Naphade, M., Natsev, A.: Multimedia semantic indexing using model vectors. In: ICME 2003, vol. 2, pp. 445–448 (2003)
Jin, Y., Khan, L., Wang, L., Awad, M.: Image annotations by combining multiple evidence & WordNet. In: ACM MM 2005, pp. 706–715 (2005)
Kennedy, L.S., Chang, S.F.: A reranking approach for context-based concept fusion in video indexing and retrieval. In: CIVR 2007, pp. 333–340 (2007)
Wang, C.H., Jing, F., Zhang, L., Zhang, H.J.: Image annotation refinement using random walk with restarts. In: ACM MM 2006, pp. 647–650 (2006)
Wang, C.H., Jing, F., Zhang, L., Zhang, H.J.: Content-based image annotation refinement. In: CVPR 2007, pp. 1–8 (2007)
Wang, P., Smeaton, A.F., Zhang, Y.C., et al.: Enhancing the detection of concepts for visual lifelogs using contexts instead of ontologies. In: ICMEW, pp. 1–6 (2014)
Shashua, A., Hazan, T.: Non-negative tensor factorization with applications to statistics and computer vision. In: ICML 2005, pp. 792–799 (2005)
Lee, D.D., Seung, H.S.: Learning the parts of objects by nonnegative matrix factorization. Nature 401, 788–791 (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Wang, P., Smeaton, A.F., Gurrin, C. (2015). Factorizing Time-Aware Multi-way Tensors for Enhancing Semantic Wearable Sensing. In: He, X., Luo, S., Tao, D., Xu, C., Yang, J., Hasan, M.A. (eds) MultiMedia Modeling. MMM 2015. Lecture Notes in Computer Science, vol 8935. Springer, Cham. https://doi.org/10.1007/978-3-319-14445-0_49
Download citation
DOI: https://doi.org/10.1007/978-3-319-14445-0_49
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-14444-3
Online ISBN: 978-3-319-14445-0
eBook Packages: Computer ScienceComputer Science (R0)