Learning Manifold Representation from Multimodal Data for Event Detection in Flickr-Like Social Media

  • Zhenguo YangEmail author
  • Qing Li
  • Wenyin Liu
  • Yun Ma
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9645)


In this work, a three-stage social event detection model is devised to discover events in Flickr data. As the features possessed by the data are typically heterogeneous, a multimodal fusion model (M\(^{2}\)F) exploits a soft-voting strategy and a reinforcing model is devised to learn fused features in the first stage. Furthermore, a Laplacian non-negative matrix factorization (LNMF) model is exploited to extract compact manifold representation. Particularly, a Laplacian regularization term constructed on the multimodal features is introduced to keep the geometry structure of the data. Finally, clustering algorithms can be applied seamlessly in order to detect event clusters. Extensive experiments conducted on the real-world dataset reveal the M\(^{2}\)F-LNMF-based approaches outperform the baselines.


Social media analytics Multimedia content analysis Multimodal fusion Manifold learning Event detection 



We would like to thank Dr. Zheng Lu, Mr. Min Cheng and Mr. Yangbin Chen for the discussions.


  1. 1.
    Ah-Pine, J., Csurka, G., Clinchant, S.: Semi-supervised visual and textual information fusion in CBMIR using graph-based methods. ACM Trans. Inf. Syst. (TOIS) 33(2), 9 (2015)CrossRefGoogle Scholar
  2. 2.
    Cai, X., Nie, F., Huang, H., Kamangar, F.: Heterogeneous image feature integration via multi-modal spectral clustering. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1977–1984. IEEE Press (2011)Google Scholar
  3. 3.
    Cai, D., He, X., Han, J., Huang, T.S.: Graph regularized nonnegative matrix factorization for data representation. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1548–1560 (2011)CrossRefGoogle Scholar
  4. 4.
    Cai, Y., Li, Q., Xie, H., Wang, T., Min, H.: Event relationship analysis for temporal event search. In: Meng, W., Feng, L., Bressan, S., Winiwarter, W., Song, W. (eds.) DASFAA 2013, Part II. LNCS, vol. 7826, pp. 179–193. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  5. 5.
    Chen, J., Cui, Y., Ye, G., Liu, D., Chang, S.F.: Event-driven semantic concept discovery by exploiting weakly tagged internet images. In: International Conference on Multimedia Retrieval, pp. 1–8. ACM (2014)Google Scholar
  6. 6.
    Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Liu, X., Huet, B.: Heterogeneous features and model selection for event-based media classification. In: 3rd ACM International Conference on Multimedia Retrieval, pp. 151–158. ACM (2013)Google Scholar
  8. 8.
    Nitta, N., Kumihashi, Y., Kato, T., Babaguchi, N.: Real-world event detection using flickr images. In: Gurrin, C., Hopfgartner, F., Hurst, W., Johansen, H., Lee, H., O’Connor, N. (eds.) MMM 2014, Part II. LNCS, vol. 8326, pp. 307–314. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  9. 9.
    Petkos, G., Papadopoulos, S., Kompatsiaris, Y.: Social event detection using multimodal clustering and integrating supervisory signals. In: 2nd ACM International Conference on Multimedia Retrieval, p. 23. ACM (2012)Google Scholar
  10. 10.
    Petkos, G., Papadopoulos, S., Mezaris, V., Kompatsiaris, Y.: Social event detection at MediaEval 2014: Challenges, datasets, and evaluation. In: MediaEval 2014 Workshop (2014)Google Scholar
  11. 11.
    Rao, Y., Li, Q.: Term weighting schemes for emerging event detection. In: 2012 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology (WI-IAT), vol. 1, pp. 105–112 (2012)Google Scholar
  12. 12.
    Sutanto, T., Nayak, R.: Ranking based clustering for social event detection. In: MediaEval 2014 Workshop, 1263, pp. 1–2 (2014)Google Scholar
  13. 13.
    Yang, Z., Li, Q., Lu, Z., Ma, Y., Gong, Z., Pan, H.: Semi-supervised multimodal clustering algorithm integrating label signals for social event detection. In: IEEE International Conference on Multimedia Big Data, pp. 32–39. IEEE (2015)Google Scholar
  14. 14.
    Yang, Z., Li, Q., Lu, Z., Ma, Y., Gong, Z., Pan, H., Chen, Y.: Semi-supervised multimodal fusion model for social event detection on web image collections. Int. J. Multimed. Data Eng. Manage. 6(4), 1–22 (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Department of Computer ScienceCity University of Hong KongHong KongChina
  2. 2.Multimedia-software Engineering Research CenterCity University of Hong KongHong KongChina

Personalised recommendations