Abstract
In this paper, we propose a new background music recommendation scheme for home videos and two new features describing the short-term motion/tempo distribution in visual/aural content. Unlike previous researches that merely matched the visual and aural contents through a perceptual way, we incorporate the textual semantics and content semantics while determining the matching degree of a video and a song. The key idea is that the recommended music should contain semantics that relate to the ones in the input video and that the rhythm of the music and the visual motion of the video should be harmonious enough. As a result, a few user-given tags and automatically annotated tags are used to compute their relation to the lyrics of the songs for selecting candidate musics. Then, we use the proposed motion-direction histogram (MDH) and pitch tempo pattern (PTP) to do the second-run selection. The user preference to the music genre is also taken into account as a filtering mechanism at the beginning. The primitive user evaluation shows that the proposed scheme is promising.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bertin-Mahieux, T., Ellis, D.P., Whitman, B., Lamere, P.: The Million Song Dataset. In: ISMIR (2011)
Chen, C.-H., Weng, M.-F., Jeng, S.-K., Chuang, Y.-Y.: Emotion-based music visualization using photos. In: Satoh, S., Nack, F., Etoh, M. (eds.) MMM 2008. LNCS, vol. 4903, pp. 358–368. Springer, Heidelberg (2008)
Chu, W.-T., Tsai, S.-Y.: Rhythm of Motion Extraction and Rhythm-Based Cross-Media Alignment for Dance Videos. IEEE Trans. MM 14(1), 129–141 (2012)
Dunker, P., Dittmar, C., Begau, A., Nowak, S., Gruhne, M.: Semantic High-Level Features for Automated Cross-Modal Slideshow Generation. In: Int. Workshop on Content-Based Multimedia Indexing, pp. 144–149 (June 2009)
Dunker, P., Popp, P., Cook, R.: Content-aware Auto-soundtracks for Personal Photo Music Slideshows. In: IEEE ICME (2011)
Feng, J., Ni, B.: Auto-Generation of Professional Background Music for Home-made Videos. In: Int. Conf. on Internet Multi. Comput. and Serv. (2010)
Goto, M.: A chorus section detection method for musical audio signals and its application to a music listening station. IEEE Trans. Audio, Speech, Lang. Process. 14(5), 1783–1794 (2006)
Li, L.-J., Su, H., Xing, E.P., Li, F.-F.: Object Bank: A High-Level Image Representation for Scene Classification & Semantic Feature Sparsification. In: NIPS, pp. 1378–1386 (2010)
Miller, G.A.: WordNet: A Lexical Database for English. Commun. ACM 38(11), 39–41 (1995)
Pampalk, E.: Computational Models of Music Similarity and Their Application in Music Information Retrieval. Ph.d., Vienna University of Technology (2006)
Wang, J., Chng, E., Xu, C.: Fully and Semi-automatic Music Sports Video Composition. In: IEEE ICME, pp. 1897–1900 (2006)
Yeh, M.-C., Cheng, K.-T.: A String Matching Approach for Visual Retrieval and Classification. In: ACM MIR, p. 52. ACM Press, New York (2008)
Zettl, H.: In: Dorai, C., Venkatesh, S. (eds.) Media Computing: Computational Media Aesthetics, pp. 11–38. Springer, US
Zhang, W., Xing, L., Huang, Q., Gao, W.: A System for Automatic Generation of Music Sports-Video. In: IEEE ICME, pp. 1286–1289 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Lin, YT., Tsai, TH., Hu, MC., Cheng, WH., Wu, JL. (2014). Semantic Based Background Music Recommendation for Home Videos. In: Gurrin, C., Hopfgartner, F., Hurst, W., Johansen, H., Lee, H., O’Connor, N. (eds) MultiMedia Modeling. MMM 2014. Lecture Notes in Computer Science, vol 8326. Springer, Cham. https://doi.org/10.1007/978-3-319-04117-9_26
Download citation
DOI: https://doi.org/10.1007/978-3-319-04117-9_26
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-04116-2
Online ISBN: 978-3-319-04117-9
eBook Packages: Computer ScienceComputer Science (R0)