Abstract
This paper describes the combination of advanced technologies for social-media-based story detection, story-based video retrieval and concept-based video (fragment) labeling under a novel approach for multimodal video annotation. This approach involves textual metadata, structural information and visual concepts - and a multimodal analytics dashboard that enables journalists to discover videos of news events, posted to social networks, in order to verify the details of the events shown. It outlines the characteristics of each individual method and describes how these techniques are blended to facilitate the content-based retrieval, discovery and summarization of (parts of) news videos. A set of case-driven experiments conducted with the help of journalists, indicate that the proposed multimodal video annotation mechanism - combined with a professional analytics dashboard which presents the collected and generated metadata about the news stories and their visual summaries - can support journalists in their content discovery and verification work.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
References
Apostolidis, E., Mezaris, V.: Fast shot segmentation combining global and local visual descriptors. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 6583–6587 (2014)
Cooray, S.H., O’Connor, N.E.: Identifying an efficient and robust sub-shot segmentation method for home movie summarisation. In: 10th International Conference on Intelligent Systems Design and Applications, pp. 1287–1292 (2010)
He, K., Zhang, X., et al.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Markatopoulou, F., Mezaris, V., et al.: Implicit and explicit concept relations in deep neural networks for multi-label video/image annotation. IEEE Trans. Circuits Syst. Video Technol. 1 (2018)
Nixon, L.J.B., Zhu, S., et al.: Video retrieval for multimedia verification of breaking news on social networks. In: 1st International Workshop on Multimedia Verification (MuVer 2017) at ACM Multimedia Conference, MuVer 2017, pp. 13–21. ACM (2017)
Over, P.D., Fiscus, J.G., et al.: TRECVID 2013-An overview of the goals, tasks, data, evaluation mechanisms and metrics. In: TRECVID 2013. NIST, USA (2013)
Pan, C.M., Chuang, Y.Y., et al.: NTU TRECVID-2007 fast rushes summarization system. In: TRECVID Workshop on Video Summarization, pp. 74–78. ACM (2007)
Pittaras, N., Markatopoulou, F., Mezaris, V., Patras, I.: Comparison of fine-tuning and extension strategies for deep convolutional neural networks. In: Amsaleg, L., Guðmundsson, G., Gurrin, C., Jónsson, B., Satoh, S. (eds.) MMM 2017. LNCS, vol. 10132, pp. 102–114. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-51811-4_9
Rublee, E., Rabaud, V., et al.: ORB: an efficient alternative to SIFT or SURF. In: 2011 International Conference on Computer Vision, pp. 2564–2571 (2011)
Russakovsky, O., Deng, J., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)
Seo, K., Park, S.J., et al.: Wipe scene-change detector based on visual rhythm spectrum. IEEE Trans. Consum. Electron. 55(2), 831–838 (2009)
Su, C.W., Tyan, H.R., et al.: A motion-tolerant dissolve detection algorithm. IEEE Int. Conf. Multimedia Expo. 2, 225–228 (2002)
Szegedy, C., Liu, W., et al.: Going deeper with convolutions. In: IEEE Conference on Computer Vision and Pattern Recognition (2015)
Teyssou, D., Leung, J.M., et al.: The InVID plug-in: web video verification on the browser. In: 1st International Workshop on Multimedia Verification (MuVer 2017) at ACM Multimedia Conference, pp. 23–30. ACM (2017)
Acknowledgments
This work was supported by the EU’s Horizon 2020 research and innovation programme under grant agreement H2020-687786 InVID.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Nixon, L., Apostolidis, E., Markatopoulou, F., Patras, I., Mezaris, V. (2019). Multimodal Video Annotation for Retrieval and Discovery of Newsworthy Video in a News Verification Scenario. In: Kompatsiaris, I., Huet, B., Mezaris, V., Gurrin, C., Cheng, WH., Vrochidis, S. (eds) MultiMedia Modeling. MMM 2019. Lecture Notes in Computer Science(), vol 11295. Springer, Cham. https://doi.org/10.1007/978-3-030-05710-7_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-05710-7_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-05709-1
Online ISBN: 978-3-030-05710-7
eBook Packages: Computer ScienceComputer Science (R0)