Cross-Media Correlation Analysis with Semi-supervised Graph Regularization
With the rapid development of multimedia data such as text, image, cross-media retrieval has become increasingly important, because users can retrieve the results with various types of media by submitting a query of any media type. The measure of relevance among different media is a basic problem. Existing methods usually only consider the original media instances (such as images, texts) but ignore their patches. In fact, cross-media patches can emphasize the important parts and improve the precision of cross-media correlation. What’s more, existing cross-media retrieval methods often focus on modeling the pairwise correlation with the similarity matrix is a constant matrix, while the similarity matrix which is not a constant matrix can improve the accuracy. In this paper, we propose a novel algorithm for cross-media data, called cross-media correlation analysis with semi-supervised graph regularization (CMCA), which can not only take full advantage of both the media instances and their patches in one graph, but also explore the similarity matrix which can improve the correlation between data. CMCA explores the sparse and semi-supervised regularization for different media types, and integrates them into a unified optimization matter, which increases the performance of the algorithm. Comparing with the current state-of-the-art methods on two datasets (i.e., Wikipedia, XMedia), the comprehensive experimental results demonstrate the effectiveness of our proposed approach.
KeywordsCross-media The measure of relevance Similarity matrix Cross-media patches
This research is supported by the National Natural Science Foundation of China (No. 61003127, No. 61373109).
- 1.Ranjan, V., Rasiwasia, N., Jawahar, C.V.: Multi-label cross-modal retrieval. In: IEEE International Conference on Computer Vision (ICCV), pp. 4094–4102 (2015)Google Scholar
- 4.Escalante, H.J., Hérnadez, C.A., Sucar, L.E., Montes, M.: Late fusion of heterogeneous methods for multimedia image retrieval. In: Proceedings of 1st ACM International Conference on Multimedia Information Retrieval, pp. 172–179 (2008)Google Scholar
- 7.Blaschko, M., Lampert, C.: Correlational spectral clustering. In: Proceedings of IEEE International Conference on Computer Vision Pattern Recognition, pp. 1–8, June 2008Google Scholar
- 8.Zhai, X., Peng, Y., Xiao, J.: Cross-modality correlation propagation for cross-media retrieval. In: Proceedings of International Conference on Acoustics, Speech, Signal Processing, pp. 2337–2340 (2012)Google Scholar
- 11.Li, D., Dimitrova, N., Li, M., Sethi, I.: Multimedia content processing through cross-modal association. In: Proceedings of ACM International Conference on Multimedia, pp. 604–611 (2003)Google Scholar
- 14.Rasiwasia, N., et al.: A new approach to cross-modal multimedia retrieval. In: Proceedings of ACM International Conference on Multimedia (ACM-MM), pp. 251–260 (2010)Google Scholar
- 15.Rasiwasia, N., Pereira, J.C., Coviello, E., Doyle, G., Lanckriet, G., Levy, R., Vasconcelos, N.: A new approach to cross-modal multimedia retrieval. In: Proceedings of 18th ACM International Conference on Multimedia, pp. 251–260 (2010)Google Scholar
- 16.Zhai, X., Peng, Y., Xiao, J.: Heterogeneous metric learning with joint graph regularization for cross-media retrieval. In: Proceedings of 27th AAAI Conference on Artificial Intelligence, pp. 1198–1204 (2013)Google Scholar
- 17.Zhai, X., Peng, Y., Xiao, J.: Cross-modality correlation propagation for cross-media retrieval. IEEE Int. Conf. Acoust. 22(10), 2337–2340 (2012)Google Scholar