Advertisement

Cross-Media Correlation Analysis with Semi-supervised Graph Regularization

  • Hong Zhang
  • Tingting Qi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10954)

Abstract

With the rapid development of multimedia data such as text, image, cross-media retrieval has become increasingly important, because users can retrieve the results with various types of media by submitting a query of any media type. The measure of relevance among different media is a basic problem. Existing methods usually only consider the original media instances (such as images, texts) but ignore their patches. In fact, cross-media patches can emphasize the important parts and improve the precision of cross-media correlation. What’s more, existing cross-media retrieval methods often focus on modeling the pairwise correlation with the similarity matrix is a constant matrix, while the similarity matrix which is not a constant matrix can improve the accuracy. In this paper, we propose a novel algorithm for cross-media data, called cross-media correlation analysis with semi-supervised graph regularization (CMCA), which can not only take full advantage of both the media instances and their patches in one graph, but also explore the similarity matrix which can improve the correlation between data. CMCA explores the sparse and semi-supervised regularization for different media types, and integrates them into a unified optimization matter, which increases the performance of the algorithm. Comparing with the current state-of-the-art methods on two datasets (i.e., Wikipedia, XMedia), the comprehensive experimental results demonstrate the effectiveness of our proposed approach.

Keywords

Cross-media The measure of relevance Similarity matrix Cross-media patches 

Notes

Acknowledgement

This research is supported by the National Natural Science Foundation of China (No. 61003127, No. 61373109).

References

  1. 1.
    Ranjan, V., Rasiwasia, N., Jawahar, C.V.: Multi-label cross-modal retrieval. In: IEEE International Conference on Computer Vision (ICCV), pp. 4094–4102 (2015)Google Scholar
  2. 2.
    Hua, Y., Wang, S., Liu, S., Cai, A., Huang, Q.: Cross-modal correlation learning by adaptive hierarchical semantic aggregation. IEEE Trans. Multimedia (TMM) 18(6), 1201–1216 (2016)CrossRefGoogle Scholar
  3. 3.
    Peng, Y., Ngo, C.-W.: Clip-based similarity measure for query-dependent clip retrieval and video summarization. IEEE Trans. Circ. Syst. Video Technol. 16(5), 612–627 (2006)CrossRefGoogle Scholar
  4. 4.
    Escalante, H.J., Hérnadez, C.A., Sucar, L.E., Montes, M.: Late fusion of heterogeneous methods for multimedia image retrieval. In: Proceedings of 1st ACM International Conference on Multimedia Information Retrieval, pp. 172–179 (2008)Google Scholar
  5. 5.
    Liu, J., Xu, C., Luo, H.: Cross-media retrieval: state-of-the-art and open issues. Int. J. Multimedia Intell. Secur. 1(1), 33–52 (2010)CrossRefGoogle Scholar
  6. 6.
    Hotelling, H.: Relations between two sets of variates. Biometrika 28(3–4), 321–377 (1936)CrossRefGoogle Scholar
  7. 7.
    Blaschko, M., Lampert, C.: Correlational spectral clustering. In: Proceedings of IEEE International Conference on Computer Vision Pattern Recognition, pp. 1–8, June 2008Google Scholar
  8. 8.
    Zhai, X., Peng, Y., Xiao, J.: Cross-modality correlation propagation for cross-media retrieval. In: Proceedings of International Conference on Acoustics, Speech, Signal Processing, pp. 2337–2340 (2012)Google Scholar
  9. 9.
    Pereira, J.C., et al.: On the role of correlation and abstraction in cross-modal multimedia retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 36(3), 521–535 (2014)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Zhai, X., Peng, Y., Xiao, J.: Learning cross-media joint representation with sparse and semisupervised regularization. IEEE Trans. Circ. Syst. Video Technol. 24(6), 965–978 (2014)CrossRefGoogle Scholar
  11. 11.
    Li, D., Dimitrova, N., Li, M., Sethi, I.: Multimedia content processing through cross-modal association. In: Proceedings of ACM International Conference on Multimedia, pp. 604–611 (2003)Google Scholar
  12. 12.
    Peng, Y., Zhai, X., Zhao, Y., Huang, X.: Semi-supervised cross-media feature learning with unified patch graph regularization. IEEE Trans. Circ. Syst. Video Technol. (TCSVT) 26(3), 583–596 (2016)CrossRefGoogle Scholar
  13. 13.
    Wang, K., He, R., Wang, L., Wang, W., Tan, T.: Joint feature selection and subspace learning for cross-modal retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 38(10), 2010–2023 (2016)CrossRefGoogle Scholar
  14. 14.
    Rasiwasia, N., et al.: A new approach to cross-modal multimedia retrieval. In: Proceedings of ACM International Conference on Multimedia (ACM-MM), pp. 251–260 (2010)Google Scholar
  15. 15.
    Rasiwasia, N., Pereira, J.C., Coviello, E., Doyle, G., Lanckriet, G., Levy, R., Vasconcelos, N.: A new approach to cross-modal multimedia retrieval. In: Proceedings of 18th ACM International Conference on Multimedia, pp. 251–260 (2010)Google Scholar
  16. 16.
    Zhai, X., Peng, Y., Xiao, J.: Heterogeneous metric learning with joint graph regularization for cross-media retrieval. In: Proceedings of 27th AAAI Conference on Artificial Intelligence, pp. 1198–1204 (2013)Google Scholar
  17. 17.
    Zhai, X., Peng, Y., Xiao, J.: Cross-modality correlation propagation for cross-media retrieval. IEEE Int. Conf. Acoust. 22(10), 2337–2340 (2012)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.College of Computer Science and TechnologyWuhan University of Science and TechnologyWuhanChina
  2. 2.Hubei Province Key Laboratory of Intelligent Information Processing and Real-Time Industrial SystemWuhanChina

Personalised recommendations