Advertisement

Combining Multi-modal Features for Social Media Analysis

  • Spiros Nikolopoulos
  • Eirini Giannakidou
  • Ioannis Kompatsiaris
  • Ioannis Patras
  • Athena Vakali

Abstract

In this chapter we discuss methods for efficiently modeling the diverse information carried by social media. The problem is viewed as a multi-modal analysis process where specialized techniques are used to overcome the obstacles arising from the heterogeneity of data. Focusing at the optimal combination of low-level features (i.e., early fusion), we present a bio-inspired algorithm for feature selection that weights the features based on their appropriateness to represent a resource. Under the same objective of optimal feature combination we also examine the use of pLSA-based aspect models, as the means to define a latent semantic space where heterogeneous types of information can be effectively combined. Tagged images taken from social sites have been used in the characteristic scenarios of image clustering and retrieval, to demonstrate the benefits of multi-modal analysis in social media.

Keywords

Feature Space Visual Word Mean Average Precision Subspace Cluster Scale Invariant Feature Transformation Descriptor 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

This work was sponsored by the European Commission as part of the Information Society Technologies (IST) programme under grant agreement no215453—WeKnowIt and the contract FP7-248984 GLOCAL.

References

  1. 1.
    Agrawal, R., Gehrke, J., Gunopulos, D., Raghavan, P.: Automatic subspace clustering of high dimensional data for data mining applications. In: Proceedings of the ACM SIGMOD Int’l Conference on Management of Data, Seattle, Washington, pp. 94–105. ACM Press, New York (1998) Google Scholar
  2. 2.
    Agrawal, R., Gehrke, J., Gunopulos, D., Raghavan, P.: Automatic subspace clustering of high dimensional data. Data Min. Knowl. Discov. 11, 5–33 (2005) CrossRefMathSciNetGoogle Scholar
  3. 3.
    Aurnhammer, M., Hanappe, P., Steels, L.: Augmenting navigation for collaborative tagging with emergent semantics. In: International Semantic Web Conference (2006) Google Scholar
  4. 4.
    Becker, H., Naaman, M., Gravano, L.: Event identification in social media. In: 12th International Workshop on the Web and Databases, WebDB (2009) Google Scholar
  5. 5.
    Becker, H., Naaman, M., Gravano, L.: Learning similarity metrics for event identification in social media. In: WSDM ’10: Proceedings of the Third ACM International Conference on Web Search and Data Mining, pp. 291–300. ACM, New York (2010) CrossRefGoogle Scholar
  6. 6.
    Blum, C.: Ant colony optimization: Introduction and recent trends. Phys. Life Rev. 2, 353–373 (2005) CrossRefGoogle Scholar
  7. 7.
    Caro, G.D., Ducatelle, F., Gambardella, L.M.: Anthocnet: an adaptive nature-inspired algorithm for routing in mobile ad hoc networks. Eur. Trans. Telecommun. 16(5), 443–455 (2005) CrossRefGoogle Scholar
  8. 8.
    Cheng, C.-H., Fu, A.W., Zhang, Y.: Entropy-based subspace clustering for mining numerical data. In: Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD ’99, pp. 84–93. ACM, New York (1999) CrossRefGoogle Scholar
  9. 9.
    Chua, T.-S., Tang, J., Hong, R., Li, H., Luo, Z., Zheng, Y.: Nus-wide: a real-world web image database from National University of Singapore. In: CIVR ’09: Proceeding of the ACM International Conference on Image and Video Retrieval, pp. 1–9. ACM, New York (2009). http://doi.acm.org/10.1145/1646396.1646452 Google Scholar
  10. 10.
    Crandall, D.J., Backstrom, L., Huttenlocher, D., Kleinberg, J.: Mapping the world’s photos. In: Proceedings of the 18th International Conference on World Wide Web. WWW ’09, pp. 761–770. ACM, New York (2009) CrossRefGoogle Scholar
  11. 11.
    Domeniconi, C., Al-Razgan, M.: Weighted cluster ensembles: Methods and analysis. ACM Trans. Knowl. Discov. Data 2, 17–11740 (2009) CrossRefGoogle Scholar
  12. 12.
    Dorigo, M.: Optimization, Learning and Natural Algorithms. Ph.D. thesis, Politecnico di Milano, Italy (1992) Google Scholar
  13. 13.
    Dorigo, M., Caro, G.D.: The ant colony optimization meta-heuristic (1999) Google Scholar
  14. 14.
    Fellbaum, C. (ed.): WordNet: An Electronic Lexical Database (Language, Speech, and Communication). MIT Press, Cambridge (1998) Google Scholar
  15. 15.
    Franz, T., Schultz, A., Sizov, S., Staab, S.: Triplerank: Ranking semantic web data by tensor decomposition. In: ISWC ’09: Proceedings of the 8th International Semantic Web Conference, pp. 213–228. Springer, Berlin (2009) Google Scholar
  16. 16.
    Giannakidou, E., Kompatsiaris, I., Vakali, A.: Semsoc: Semantic, social and content-based clustering in multimedia collaborative tagging systems. In: ICSC, pp. 128–135 (2008) Google Scholar
  17. 17.
    Giannakidou, E., Koutsonikola, V.A., Vakali, A., Kompatsiaris, Y.: Co-clustering tags and social data sources. In: WAIM, pp. 317–324 (2008) Google Scholar
  18. 18.
    Harshman, R.A., Lundy, M.E.: Parafac: Parallel factor analysis. Comput. Stat. Data Anal. 18(1), 39–72 (1994) CrossRefMATHGoogle Scholar
  19. 19.
    Hofmann, T.: Unsupervised learning from dyadic data. In: NJPS, pp. 466–472. MIT Press, Cambridge (1998) Google Scholar
  20. 20.
    Hofmann, T.: Probabilistic latent semantic analysis. In: Proc. of Uncertainty in Artificial Intelligence, UAI’99, Stockholm (1999). URL citeseer.ist.psu.edu/hofmann99probabilistic.html
  21. 21.
    Kennedy, L., Naaman, M.: Less talk, more rock: automated organization of community-contributed collections of concert videos. In: Proceedings of the 18th International Conference on World Wide Web. WWW ’09, pp. 311–320. ACM, New York (2009) CrossRefGoogle Scholar
  22. 22.
    Kennedy, L.S., Naaman, M., Ahern, S., Nair, R., Rattenbury, T.: How flickr helps us make sense of the world: context and content in community-contributed media collections. In: ACM Multimedia, pp. 631–640 (2007) Google Scholar
  23. 23.
    Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51(3), 455–500 (2009). doi: 10.1137/07070111X CrossRefMATHMathSciNetGoogle Scholar
  24. 24.
    Lathauwer, L.D., Moor, B.D., Vandewalle, J.: A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. 21(4), 1253–1278 (2000) CrossRefMATHMathSciNetGoogle Scholar
  25. 25.
    Li, D., Dimitrova, N., Li, M., Sethi, I.K.: Multimedia content processing through cross-modal association. In: MULTIMEDIA ’03, pp. 604–611. ACM, New York (2003) CrossRefGoogle Scholar
  26. 26.
    Lienhart, R., Romberg, S., Hörster, E.: Multilayer plsa for multimodal image retrieval. In: CIVR ’09: Proceeding of the ACM International Conference on Image and Video Retrieval, pp. 1–8. ACM, New York (2009). http://doi.acm.org/10.1145/1646396.1646408 CrossRefGoogle Scholar
  27. 27.
    Lindstaedt, S., Pammer, V., Mörzinger, R., Kern, R., Mülner, H., Wagner, C.: Recommending tags for pictures based on text, visual content and user context. In: Proceedings of the 2008 Third International Conference on Internet and Web Applications and Services, pp. 506–511. IEEE Computer Society, Washington (2008) CrossRefGoogle Scholar
  28. 28.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004) CrossRefGoogle Scholar
  29. 29.
    Magalhaes, J., Rüger, S.: Information-theoretic semantic multimedia indexing. In: CIVR ’07, pp. 619–626. ACM, New York (2007). http://doi.acm.org/10.1145/1282280.1282368 Google Scholar
  30. 30.
    Manjunath, B.S., Ohm, J.R., Vinod, V.V., Yamada, A.: Colour and texture descriptors. IEEE Trans. Circuits Syst. Video Technol., Special Issue on MPEG-7 11(6), 703–715 (2001) CrossRefGoogle Scholar
  31. 31.
    MPEG-7: Visual Experimentation Model (XM). Version 10.0, ISO/IEC/JTC1/SC29/WG11, Doc. N4062 (2001) Google Scholar
  32. 32.
    Olivares, X., Ciaramita, M., van Zwol, R.: Boosting image retrieval through aggregating search results based on visual annotations. In: Proceeding of the 16th ACM International Conference on Multimedia. MM ’08, pp. 189–198. ACM, New York (2008) CrossRefGoogle Scholar
  33. 33.
    Parsons, L., Haque, E., Liu, H.: Subspace clustering for high dimensional data: a review. SIGKDD Explor. Newsl. 6, 90–105 (2004) CrossRefGoogle Scholar
  34. 34.
    Piatrik, T., Izquierdo, E.: Subspace clustering of images using ant colony optimisation. In: 16th IEEE International Conference on Image Processing (ICIP), pp. 229–232 (2009) CrossRefGoogle Scholar
  35. 35.
    Quack, T., Leibe, B., Gool, L.J.V.: World-scale mining of objects and events from community photo collections. In: CIVR, pp. 47–56 (2008) CrossRefGoogle Scholar
  36. 36.
    Sigurbjörnsson, B., van Zwol, R.: Flickr tag recommendation based on collective knowledge. In: Proceeding of the 17th International Conference on World Wide Web. WWW ’08, pp. 327–336. ACM, New York (2008) CrossRefGoogle Scholar
  37. 37.
    Sivic, J., Zisserman, A.: Video google: A text retrieval approach to object matching in videos. In: ICCV ’03: Proceedings of the Ninth IEEE International Conference on Computer Vision, p. 1470. IEEE Computer Society, Washington (2003) CrossRefGoogle Scholar
  38. 38.
    Sizov, S.: Geofolk: latent spatial semantics in web 2.0 social media. In: WSDM ’10: Proceedings of the Third ACM International Conference on Web Search and Data Mining, pp. 281–290. ACM, New York (2010). http://doi.acm.org/10.1145/1718487.1718522 CrossRefGoogle Scholar
  39. 39.
    Symeonidis, P., Nanopoulos, A., Manolopoulos, Y.: Tag recommendations based on tensor dimensionality reduction. In: RecSys ’08: Proceedings of the 2008 ACM Conference on Recommender Systems, pp. 43–50. ACM, New York (2008) CrossRefGoogle Scholar
  40. 40.
    Wu, Y., Chang, E.Y., Chang, K.C.-C., Smith, J.R.: Optimal multimodal fusion for multimedia data analysis. In: MULTIMEDIA ’04, pp. 572–579. ACM, New York (2004) CrossRefGoogle Scholar
  41. 41.
    Xu, R., Wunsch, I.: Survey of clustering algorithms. IEEE Trans. Neural Netw. 16(3), 645–678 (2005) CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  • Spiros Nikolopoulos
    • 1
    • 2
  • Eirini Giannakidou
    • 1
    • 3
  • Ioannis Kompatsiaris
    • 1
  • Ioannis Patras
    • 2
  • Athena Vakali
    • 3
  1. 1.Informatics & Telematics InstituteThermi, ThessalonikiGreece
  2. 2.School of Electronic Engineering and Computer ScienceQueen Mary University of LondonLondonUK
  3. 3.Department of Computer ScienceAristotle University of ThessalonikiThessalonikiGreece

Personalised recommendations