Advertisement

User-Perceptive Multimedia Content Analysis

  • Jitao SangEmail author
Chapter
  • 415 Downloads
Part of the Springer Theses book series (Springer Theses)

Abstract

Typical social multimedia services allow users as uploaders, viewers, taggers, and commenters to interact and collaborate with each other in a communication dialog. The wisdom of crowds provides a huge resource for understanding social multimedia content. In this chapter, we explicitly model user interaction in the tag generation process and propose a regularized tensor factorization solution to refine the ternary correlations among user, image, and tag. While the traditional social tag analysis work focus on analyzing the image-tag binary correlation, taking user factor into consideration shows superior performance in image tag refinement task.

Keywords

Tensor Factorization Smoothness Constraint Latent Subspace Random Walk With Restart Tucker Decomposition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Acar, E., Yener, B.: Unsupervised multiway data analysis: a literature survey. IEEE Trans. Knowl. Data Eng. 21(1), 6–20 (2009)CrossRefGoogle Scholar
  2. 2.
    Bengio, Y., Paiement, J.-F., Vincent, P., Delalleau, O., Roux, N.L., Ouimet, M.: Out-of-sample extensions for lle, isomap, mds, eigenmaps, and spectral clustering. In: NIPS (2003)Google Scholar
  3. 3.
    Borghol, Y., Ardon, S., Carlsson, N., Eager, D., Mahanti, A.: The untold story of the clones: content-agnostic factors that impact youtube video popularity. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD’12, pp. 1186–1194 (2012)Google Scholar
  4. 4.
    Chen, L., Xu, D., Tsang, I.W.-H., Luo, J.: Tag-based web photo retrieval improved by batch mode re-tagging. In: CVPR, pp. 3440–3446 (2010)Google Scholar
  5. 5.
    Chua, T.-S., Tang, J., Hong, R., Li, H., Luo, Z., Zheng, Y.: Nus-wide: a real-world web image database from national university of singapore. In: CIVR (2009)Google Scholar
  6. 6.
    Cranshaw, J., Schwartz, R., Hong, J.I., Sadeh, N.M.: The livehoods project: utilizing social media to understand the dynamics of a city. In: ICWSM (2012)Google Scholar
  7. 7.
    De Choudhury, M., Sundaram, H., John, A., Seligmann, D.D.: What makes conversations interesting? Themes, participants and consequences of conversations in online social media. In: Proceedings of the 18th International Conference on World Wide Web, WWW’09, pp. 331–340 (2009)Google Scholar
  8. 8.
    Eickhoff, C., Li, W., de Vries, A.P.: Exploiting user comments for audio-visual content indexing and retrieval. In: 34th European Conference on Information Retrieval (ECIR) (2013)Google Scholar
  9. 9.
    Fang, Q., Sang, J., Xu, C., Rui, Y.: Topic-sensitive influencer mining in interest-based social media networks via hypergraph learning. IEEE Trans. Multimed. 16(3), 796–812 (2014)CrossRefGoogle Scholar
  10. 10.
    Feng, W., Wang, J.: Incorporating heterogeneous information for personalized tag recommendation in social tagging systems. In: KDD, pp. 1276–1284 (2012)Google Scholar
  11. 11.
    Filippova, K., Hall, K.B.: Improved video categorization from text metadata and user comments. In: Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR’11, pp. 835–842 (2011)Google Scholar
  12. 12.
    He, X., Kan, M.-Y., Xie, P., Chen, X.: Comment-based multi-view clustering of web 2.0 items. In: Proceedings of the 23rd International Conference on World Wide Web, WWW’14, pp. 771–782 (2014)Google Scholar
  13. 13.
    Helic, D.,Strohmaier, M.: Building directories for social tagging systems. In: Proceedings of the 20th ACM International Conference on Information and Knowledge Management, CIKM’10, pp. 525–534 (2011)Google Scholar
  14. 14.
    Hu, X., Tang, L., Tang, J., Liu, H.: Exploiting social relations for sentiment analysis in microblogging. In: WSDM, pp. 537–546 (2013)Google Scholar
  15. 15.
    Jin, X., Wang, C., Luo, J., Yu, X., Han, J.: Likeminer: a system for mining the power of ‘like’ in social media networks. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD’11, pp. 753–756 (2011)Google Scholar
  16. 16.
    Jin, Y., Khan, L., Wang, L., Awad, M.: Image annotations by combining multiple evidence & wordnet. In: ACM Multimedia, pp. 706–715 (2005)Google Scholar
  17. 17.
    Lappas, T., Punera, K., Sarlos, T.: Mining tags using social endorsement networks. In: Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR’11, pp. 195–204 (2011)Google Scholar
  18. 18.
    Li, W.-J., Yeung, D.-Y.: Relation regularized matrix factorization. In: IJCAI, pp. 1126–1131 (2009)Google Scholar
  19. 19.
    Li, Z., Liu, J., Zhu, X., Liu, T., Lu, H.: Image annotation using multi-correlation probabilistic matrix factorization. In: ACM Multimedia, pp. 1187–1190 (2010)Google Scholar
  20. 20.
    Liu, D., Hua, X.-S., Wang, M., Zhang, H.-J.: Image retagging. In: ACM Multimedia, pp. 491–500 (2010)Google Scholar
  21. 21.
    Liu, D., Hua, X.-S., Yang, L., Wang, M., Zhang, H.-J.: Tag ranking. In: WWW, pp. 351–360 (2009)Google Scholar
  22. 22.
    Liu, D., Hua, X.-S., Zhang, H.-J.: Content-based tag processing for internet social images. Multimed. Tool. Appl. 51, 723–738 (2011)CrossRefGoogle Scholar
  23. 23.
    Liu, D., Yan, S., Rui, Y., Zhang, H.-J.: Unified tag analysis with multi-edge graph. In: ACM Multimedia, pp. 25–34 (2010)Google Scholar
  24. 24.
    Liu, J., Wang, B., Li, M., Li, Z., Ma, W.-Y., Lu, H., Ma, S.: Dual cross-media relevance model for image annotation. In: ACM Multimedia, pp. 605–614 (2007)Google Scholar
  25. 25.
    Liu, X., Yan, S., Cheng, B., Tang, J., Chua, T.-S., Jin, H.: Label-to-region with continuity-biased bi-layer sparsity priors. ACM Trans. Multimed. Comput. Commun. Appl. (TOMCCAP) 8(4), 50 (2012)Google Scholar
  26. 26.
    Lu, C., Hu, X., Chen, X., Park, J.-R., He, T., Li, Z.: The topic-perspective model for social tagging systems. In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 683–692 (2010)Google Scholar
  27. 27.
    man Au Yeung, C., Gibbins, N., Shadbolt, N.: A study of user profile generation from folksonomies. In: SWKM (2008)Google Scholar
  28. 28.
    Pinto, H., Almeida, J.M., Gonçalves, M.A.: Using early view patterns to predict the popularity of youtube videos. In: Proceedings of the Sixth ACM International Conference on Web Search and Data Mining, WSDM’13, pp. 365–374 (2013)Google Scholar
  29. 29.
    Plangprasopchok, A., Lerman, K., Getoor, L.: Growing a tree in the forest: Constructing folksonomies by integrating structured metadata. In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD’10, pp. 949–958 (2010)Google Scholar
  30. 30.
    Potthast, M., Stein, B., Becker, S.: Towards comment-based cross-media retrieval. In: Proceedings of the 19th International Conference on World Wide Web, WWW’10, pp. 1169–1170 (2010)Google Scholar
  31. 31.
    Rendle, S., Marinho, L.B., Nanopoulos, A., Schmidt-Thieme, L.: Learning optimal ranking with tensor factorization for tag recommendation. In: KDD, pp. 727–736 (2009)Google Scholar
  32. 32.
    Rendle, S., Schmidt-Thieme, L.: Pairwise interaction tensor factorization for personalized tag recommendation. In: WSDM, pp. 81–90 (2010)Google Scholar
  33. 33.
    Sang, J., Liu, J., Xu, C.: Exploiting user information for image tag refinement. In: ACM Multimedia, pp. 1129–1132 (2011)Google Scholar
  34. 34.
    Sang, J., Xu, C., Liu, J.: User-aware image tag refinement via ternary semantic analysis. IEEE Trans. Multimed. 14(3–2), 883–895 (2012)CrossRefGoogle Scholar
  35. 35.
    Sang, J., Xu, C., Lu, D.: Learn to personalized image search from the photo sharing websites. IEEE Trans. Multimed. 14(4), 963–974 (2012)CrossRefGoogle Scholar
  36. 36.
    Siersdorfer, S., Chelaru, S., Nejdl, W., San Pedro, J.: How useful are your comments? Analyzing and predicting youtube comments and comment ratings. In: Proceedings of the 19th International Conference on World Wide Web, WWW’10, pp. 891–900 (2010)Google Scholar
  37. 37.
    Trevisiol, M., Jégou, H., Delhumeau, J., Gravier, G.: Retrieving geo-location of videos with a divide & conquer hierarchical multimodal approach. In: ICMR, pp. 1–8 (2013)Google Scholar
  38. 38.
    von Ahn, L., Dabbish, L.: Esp: Labeling images with a computer game. In: AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors, pp. 91–98 (2005)Google Scholar
  39. 39.
    Wang, C., Jing, F., Zhang, L., Zhang, H.: Image annotation refinement using random walk with restarts. In: ACM Multimedia, pp. 647–650 (2006)Google Scholar
  40. 40.
    Wang, C., Jing, F., Zhang, L., Zhang, H.-J.: Content-based image annotation refinement. In: CVPR (2007)Google Scholar
  41. 41.
    Xie, L., Natsev, A., Hill, M.L., Smith, J.R., Phillips, A.: The accuracy and value of machine-generated image tags: design and user evaluation of an end-to-end image tagging system. In: CIVR, pp. 58–65 (2010)Google Scholar
  42. 42.
    Xu, H., Wang, J., Hua, X.-S., Li, S.: Tag refinement by regularized lda. In: ACM Multimedia, pp. 573–576 (2009)Google Scholar
  43. 43.
    Yamamoto, T., Nakamura, S.: Leveraging viewer comments for mood classification of music video clips. In: Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR’13, pp. 797–800 (2013)Google Scholar
  44. 44.
    Ye, M., Shou, D., Lee, W.-C., Yin, P., Janowicz, K.: On the semantic annotation of places in location-based social networks. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD’11, pp. 520–528 (2011)Google Scholar
  45. 45.
    Yu, B., Ma, W.-Y., Nahrstedt, K., Zhang, H.-J.: Video summarization based on user log enhanced link analysis. In: Proceedings of the Eleventh ACM International Conference on Multimedia, MULTIMEDIA’03, pp. 382–391 (2003)Google Scholar
  46. 46.
    Zhou, Y., Wilkinson, D.M., Schreiber, R., Pan, R.: Large-scale parallel collaborative filtering for the netflix prize. In: AAIM, pp. 337–348 (2008)Google Scholar
  47. 47.
    Zhu, G., Yan, S., Ma, Y.: Image tag refinement towards low-rank, content-tag prior and error sparsity. In: ACM Multimedia, pp. 461–470 (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.National Lab of Pattern Recognition, Institute of AutomationChinese Academy of SciencesBeijingChina

Personalised recommendations