Skip to main content
SpringerLink
Log in

Combining Multi-modal Features for Social Media Analysis

  • Chapter
Social Media Modeling and Computing

Abstract

In this chapter we discuss methods for efficiently modeling the diverse information carried by social media. The problem is viewed as a multi-modal analysis process where specialized techniques are used to overcome the obstacles arising from the heterogeneity of data. Focusing at the optimal combination of low-level features (i.e., early fusion), we present a bio-inspired algorithm for feature selection that weights the features based on their appropriateness to represent a resource. Under the same objective of optimal feature combination we also examine the use of pLSA-based aspect models, as the means to define a latent semantic space where heterogeneous types of information can be effectively combined. Tagged images taken from social sites have been used in the characteristic scenarios of image clustering and retrieval, to demonstrate the benefits of multi-modal analysis in social media.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    As many users find the tagging process tedious, the scenario that most photos in each group have been assigned only one tag is not far from reality.

  2. 2.

    For Flickr resources and metadata download the Flickr API along with the utility wget were used.

  3. 3.

    http://lms.comp.nus.edu.sg/research/NUS-WIDE.htm.

  4. 4.

    http://www.flickr.com/.

  5. 5.

    http://www.delicious.com/.

  6. 6.

    http://www.last.fm.

References

  1. Agrawal, R., Gehrke, J., Gunopulos, D., Raghavan, P.: Automatic subspace clustering of high dimensional data for data mining applications. In: Proceedings of the ACM SIGMOD Int’l Conference on Management of Data, Seattle, Washington, pp. 94–105. ACM Press, New York (1998)

    Google Scholar 

  2. Agrawal, R., Gehrke, J., Gunopulos, D., Raghavan, P.: Automatic subspace clustering of high dimensional data. Data Min. Knowl. Discov. 11, 5–33 (2005)

    Article  MathSciNet  Google Scholar 

  3. Aurnhammer, M., Hanappe, P., Steels, L.: Augmenting navigation for collaborative tagging with emergent semantics. In: International Semantic Web Conference (2006)

    Google Scholar 

  4. Becker, H., Naaman, M., Gravano, L.: Event identification in social media. In: 12th International Workshop on the Web and Databases, WebDB (2009)

    Google Scholar 

  5. Becker, H., Naaman, M., Gravano, L.: Learning similarity metrics for event identification in social media. In: WSDM ’10: Proceedings of the Third ACM International Conference on Web Search and Data Mining, pp. 291–300. ACM, New York (2010)

    Chapter  Google Scholar 

  6. Blum, C.: Ant colony optimization: Introduction and recent trends. Phys. Life Rev. 2, 353–373 (2005)

    Article  Google Scholar 

  7. Caro, G.D., Ducatelle, F., Gambardella, L.M.: Anthocnet: an adaptive nature-inspired algorithm for routing in mobile ad hoc networks. Eur. Trans. Telecommun. 16(5), 443–455 (2005)

    Article  Google Scholar 

  8. Cheng, C.-H., Fu, A.W., Zhang, Y.: Entropy-based subspace clustering for mining numerical data. In: Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD ’99, pp. 84–93. ACM, New York (1999)

    Chapter  Google Scholar 

  9. Chua, T.-S., Tang, J., Hong, R., Li, H., Luo, Z., Zheng, Y.: Nus-wide: a real-world web image database from National University of Singapore. In: CIVR ’09: Proceeding of the ACM International Conference on Image and Video Retrieval, pp. 1–9. ACM, New York (2009). http://doi.acm.org/10.1145/1646396.1646452

    Google Scholar 

  10. Crandall, D.J., Backstrom, L., Huttenlocher, D., Kleinberg, J.: Mapping the world’s photos. In: Proceedings of the 18th International Conference on World Wide Web. WWW ’09, pp. 761–770. ACM, New York (2009)

    Chapter  Google Scholar 

  11. Domeniconi, C., Al-Razgan, M.: Weighted cluster ensembles: Methods and analysis. ACM Trans. Knowl. Discov. Data 2, 17–11740 (2009)

    Article  Google Scholar 

  12. Dorigo, M.: Optimization, Learning and Natural Algorithms. Ph.D. thesis, Politecnico di Milano, Italy (1992)

    Google Scholar 

  13. Dorigo, M., Caro, G.D.: The ant colony optimization meta-heuristic (1999)

    Google Scholar 

  14. Fellbaum, C. (ed.): WordNet: An Electronic Lexical Database (Language, Speech, and Communication). MIT Press, Cambridge (1998)

    Google Scholar 

  15. Franz, T., Schultz, A., Sizov, S., Staab, S.: Triplerank: Ranking semantic web data by tensor decomposition. In: ISWC ’09: Proceedings of the 8th International Semantic Web Conference, pp. 213–228. Springer, Berlin (2009)

    Google Scholar 

  16. Giannakidou, E., Kompatsiaris, I., Vakali, A.: Semsoc: Semantic, social and content-based clustering in multimedia collaborative tagging systems. In: ICSC, pp. 128–135 (2008)

    Google Scholar 

  17. Giannakidou, E., Koutsonikola, V.A., Vakali, A., Kompatsiaris, Y.: Co-clustering tags and social data sources. In: WAIM, pp. 317–324 (2008)

    Google Scholar 

  18. Harshman, R.A., Lundy, M.E.: Parafac: Parallel factor analysis. Comput. Stat. Data Anal. 18(1), 39–72 (1994)

    Article  MATH  Google Scholar 

  19. Hofmann, T.: Unsupervised learning from dyadic data. In: NJPS, pp. 466–472. MIT Press, Cambridge (1998)

    Google Scholar 

  20. Hofmann, T.: Probabilistic latent semantic analysis. In: Proc. of Uncertainty in Artificial Intelligence, UAI’99, Stockholm (1999). URL citeseer.ist.psu.edu/hofmann99probabilistic.html

  21. Kennedy, L., Naaman, M.: Less talk, more rock: automated organization of community-contributed collections of concert videos. In: Proceedings of the 18th International Conference on World Wide Web. WWW ’09, pp. 311–320. ACM, New York (2009)

    Chapter  Google Scholar 

  22. Kennedy, L.S., Naaman, M., Ahern, S., Nair, R., Rattenbury, T.: How flickr helps us make sense of the world: context and content in community-contributed media collections. In: ACM Multimedia, pp. 631–640 (2007)

    Google Scholar 

  23. Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51(3), 455–500 (2009). doi:10.1137/07070111X

    Article  MATH  MathSciNet  Google Scholar 

  24. Lathauwer, L.D., Moor, B.D., Vandewalle, J.: A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. 21(4), 1253–1278 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  25. Li, D., Dimitrova, N., Li, M., Sethi, I.K.: Multimedia content processing through cross-modal association. In: MULTIMEDIA ’03, pp. 604–611. ACM, New York (2003)

    Chapter  Google Scholar 

  26. Lienhart, R., Romberg, S., Hörster, E.: Multilayer plsa for multimodal image retrieval. In: CIVR ’09: Proceeding of the ACM International Conference on Image and Video Retrieval, pp. 1–8. ACM, New York (2009). http://doi.acm.org/10.1145/1646396.1646408

    Chapter  Google Scholar 

  27. Lindstaedt, S., Pammer, V., Mörzinger, R., Kern, R., Mülner, H., Wagner, C.: Recommending tags for pictures based on text, visual content and user context. In: Proceedings of the 2008 Third International Conference on Internet and Web Applications and Services, pp. 506–511. IEEE Computer Society, Washington (2008)

    Chapter  Google Scholar 

  28. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  29. Magalhaes, J., Rüger, S.: Information-theoretic semantic multimedia indexing. In: CIVR ’07, pp. 619–626. ACM, New York (2007). http://doi.acm.org/10.1145/1282280.1282368

    Google Scholar 

  30. Manjunath, B.S., Ohm, J.R., Vinod, V.V., Yamada, A.: Colour and texture descriptors. IEEE Trans. Circuits Syst. Video Technol., Special Issue on MPEG-7 11(6), 703–715 (2001)

    Article  Google Scholar 

  31. MPEG-7: Visual Experimentation Model (XM). Version 10.0, ISO/IEC/JTC1/SC29/WG11, Doc. N4062 (2001)

    Google Scholar 

  32. Olivares, X., Ciaramita, M., van Zwol, R.: Boosting image retrieval through aggregating search results based on visual annotations. In: Proceeding of the 16th ACM International Conference on Multimedia. MM ’08, pp. 189–198. ACM, New York (2008)

    Chapter  Google Scholar 

  33. Parsons, L., Haque, E., Liu, H.: Subspace clustering for high dimensional data: a review. SIGKDD Explor. Newsl. 6, 90–105 (2004)

    Article  Google Scholar 

  34. Piatrik, T., Izquierdo, E.: Subspace clustering of images using ant colony optimisation. In: 16th IEEE International Conference on Image Processing (ICIP), pp. 229–232 (2009)

    Chapter  Google Scholar 

  35. Quack, T., Leibe, B., Gool, L.J.V.: World-scale mining of objects and events from community photo collections. In: CIVR, pp. 47–56 (2008)

    Chapter  Google Scholar 

  36. Sigurbjörnsson, B., van Zwol, R.: Flickr tag recommendation based on collective knowledge. In: Proceeding of the 17th International Conference on World Wide Web. WWW ’08, pp. 327–336. ACM, New York (2008)

    Chapter  Google Scholar 

  37. Sivic, J., Zisserman, A.: Video google: A text retrieval approach to object matching in videos. In: ICCV ’03: Proceedings of the Ninth IEEE International Conference on Computer Vision, p. 1470. IEEE Computer Society, Washington (2003)

    Chapter  Google Scholar 

  38. Sizov, S.: Geofolk: latent spatial semantics in web 2.0 social media. In: WSDM ’10: Proceedings of the Third ACM International Conference on Web Search and Data Mining, pp. 281–290. ACM, New York (2010). http://doi.acm.org/10.1145/1718487.1718522

    Chapter  Google Scholar 

  39. Symeonidis, P., Nanopoulos, A., Manolopoulos, Y.: Tag recommendations based on tensor dimensionality reduction. In: RecSys ’08: Proceedings of the 2008 ACM Conference on Recommender Systems, pp. 43–50. ACM, New York (2008)

    Chapter  Google Scholar 

  40. Wu, Y., Chang, E.Y., Chang, K.C.-C., Smith, J.R.: Optimal multimodal fusion for multimedia data analysis. In: MULTIMEDIA ’04, pp. 572–579. ACM, New York (2004)

    Chapter  Google Scholar 

  41. Xu, R., Wunsch, I.: Survey of clustering algorithms. IEEE Trans. Neural Netw. 16(3), 645–678 (2005)

    Article  Google Scholar 

Download references

Acknowledgements

This work was sponsored by the European Commission as part of the Information Society Technologies (IST) programme under grant agreement no215453—WeKnowIt and the contract FP7-248984 GLOCAL.

Author information

Authors and Affiliations

  1. Informatics & Telematics Institute, Thermi, Thessaloniki, Greece

    Spiros Nikolopoulos, Eirini Giannakidou & Ioannis Kompatsiaris

  2. School of Electronic Engineering and Computer Science, Queen Mary University of London, E1 4NS, London, UK

    Spiros Nikolopoulos & Ioannis Patras

  3. Department of Computer Science, Aristotle University of Thessaloniki, Thessaloniki, Greece

    Eirini Giannakidou & Athena Vakali

Authors
  1. Spiros Nikolopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eirini Giannakidou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ioannis Kompatsiaris
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ioannis Patras
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Athena Vakali
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Spiros Nikolopoulos .

Editor information

Editors and Affiliations

  1. School of Computer Engineering, Nanyang Technological University, Singapore, 639798, Singapore

    Steven C.H. Hoi

  2. Kodak Research Laboratories, Lake Avenue 1999, Rochester, 14650, New York, USA

    Jiebo Luo

  3. Media Informatics and Multimedia Systems, University of Oldenburg, Escherweg 2, Oldenburg, 26121, Germany

    Susanne Boll

  4. School of Computer Engineering, Nanyang Technological University, Singapore, 639798, Singapore

    Dong Xu

  5. Dept. Computer Science and Engineering, Michigan State University, Engineering Building 3115, East Lansing, 48824, Michigan, USA

    Rong Jin

  6. Dept. Computer Science and Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong/PR China

    Irwin King

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag London Limited

About this chapter

Cite this chapter

Nikolopoulos, S., Giannakidou, E., Kompatsiaris, I., Patras, I., Vakali, A. (2011). Combining Multi-modal Features for Social Media Analysis. In: Hoi, S., Luo, J., Boll, S., Xu, D., Jin, R., King, I. (eds) Social Media Modeling and Computing. Springer, London. https://doi.org/10.1007/978-0-85729-436-4_4

Download citation

  • DOI: https://doi.org/10.1007/978-0-85729-436-4_4

  • Publisher Name: Springer, London

  • Print ISBN: 978-0-85729-435-7

  • Online ISBN: 978-0-85729-436-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation