Review of Methods to Predict Social Image Interestingness and Memorability

  • Xesca AmengualEmail author
  • Anna Bosch
  • Josep Lluís de la Rosa
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9256)


An entire industry has developed around keyword optimization for ad buyers. However, social media landscape has shift to a photo driven behavior and there is a need to overcome the challenge to analyze all this large amount of visual data that users post in internet. We will address this analysis by providing a review on how to measure image and video interestingness and memorability from content that is tacked spontaneously in social networks. We will investigate current state-of-the-art of methods analyzing social media images and provide further research directions that could be beneficial for both, users and companies.


Interestingness Memorability Image Video Review 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Pang, B., Lee, L.: Opinion mining and sentiment analysis. Found. Trends Inf. Retr. 2(1–2), 1–135 (2008)CrossRefGoogle Scholar
  2. 2.
    Bifet, A., Frank, E.: Sentiment knowledge discovery in twitter streaming data. In: Pfahringer, B., Holmes, G., Hoffmann, A. (eds.) DS 2010. LNCS, vol. 6332, pp. 1–15. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  3. 3.
    Liu, B., Hu, M., Cheng, J.: Opinion observer: analyzing and comparing opinions on the web. In: 14th int. conf. on World Wide Web, pp. 342–351. ACM (2005)Google Scholar
  4. 4.
    Mitrović, M., Paltoglou, G., Tadić, B.: Quantitative analysis of bloggers’ collective behavior powered by emotions. J. Stat. Mech: Theory Exp. (02), P02005 (2011)Google Scholar
  5. 5.
    Dhar, S., Ordonez, V., Berg, T.L.: High level describable attributes for predicting aesthetics and interestingness. In: CVPR, pp. 1657–1664. IEEE (2011)Google Scholar
  6. 6.
    Isola, P., Xiao, J., Torralba, A., Oliva, A.: What makes an image memorable? In: CVPR, pp. 145–152. IEEE (2011)Google Scholar
  7. 7.
    Silberschatz, A., Tuzhilin, A.: On subjective measures of interestingness in knowledge discovery. In: KDD, pp. 275–281 (1995)Google Scholar
  8. 8.
    Tan, P.N., Kumar, V., Srivastava, J.: Selecting the right interestingness measure for association patterns. In: ACM SIGKDD, pp. 32–41. (2002)Google Scholar
  9. 9.
    Schaul, T., Pape, L., Glasmachers, T., Graziano, V., Schmidhuber, J.: Coherence progress: a measure of interestingness based on fixed compressors. In: Schmidhuber, J., Thórisson, K.R., Looks, M. (eds.) AGI 2011. LNCS, vol. 6830, pp. 21–30. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  10. 10.
    Berlyne, D.E.: Conflict, arousal, and curiosity (1960)Google Scholar
  11. 11.
    Chen, A., Darst, P.W., Pangrazi, R.P.: An examination of situational interest and its sources. British Journal of Educational Psychology 71(3), 383–400 (2001)CrossRefGoogle Scholar
  12. 12.
    Smith, C.A., Ellsworth, P.C.: Patterns of cognitive appraisal in emotion. Journal of Personality and Social Psychology 48(4), 813 (1985)CrossRefGoogle Scholar
  13. 13.
    Turner Jr, S.A., Silvia, P.J.: Must interesting things be pleasant? a test of competing appraisal structures. Emotion 6(4), 670 (2006)CrossRefGoogle Scholar
  14. 14.
    Grabner, H., Nater, F., Druey, M., Van Gool, L.: Visual interestingness in image sequences. In: ACM Int. Conf. Multimed., pp. 1017–1026 (2013)Google Scholar
  15. 15.
    Gygli, M., Grabner, H., Riemenschneider, H., Nater, F., Gool, L.V.: The interestingness of images. In: ICCV, pp. 1633–1640. IEEE (2013)Google Scholar
  16. 16.
    Fu, Y., Hospedales, T.M., Xiang, T., Gong, S., Yao, Y.: Interestingness prediction by robust learning to rank. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part II. LNCS, vol. 8690, pp. 488–503. Springer, Heidelberg (2014) Google Scholar
  17. 17.
    Hsieh, L.C., Hsu, W.H., Wang, H.C.: Investigating and predicting social and visual image interestingness on social media by crowdsourcing. In: ICASSP IEEE (2014)Google Scholar
  18. 18.
    Fiolet, E.: Analyzing image popularityGoogle Scholar
  19. 19.
    Khosla, A., Das Sarma, A., Hamid, R.: What makes an image popular? In: World Wide Web, pp. 867–876 (2014)Google Scholar
  20. 20.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42(3), 145–175 (2001)CrossRefzbMATHGoogle Scholar
  21. 21.
    Jiang, Y.G., Wang, Y., Feng, R., Xue, X., Zheng, Y., Yang, H.: Understanding and predicting interestingness of videos. In: AAAI. (2013)Google Scholar
  22. 22.
    Judd, T., Ehinger, K., Durand, F., Torralba, A.: Learning to predict where humans look. In: Computer Vision, pp. 2106–2113. IEEE (2009)Google Scholar
  23. 23.
    Spain, M., Perona, P.: Measuring and predicting importance of objects in our visual world. Tech. Rep. (2007)Google Scholar
  24. 24.
    Berg, A.C., Berg, T.L., Daume, H., Dodge, J., Goyal, A., Han, X., et al.: Understanding and predicting importance in images. In: CVPR, pp. 3562–3569. IEEE (2012)Google Scholar
  25. 25.
    Turakhia, N., Parikh, D.: Attribute dominance: what pops out? In: ICCV, pp. 1225–1232. IEEE (2013)Google Scholar
  26. 26.
    Isola, P., Parikh, D., Torralba, A., Oliva, A.: Understanding the intrinsic memorability of images. In: NIPS, pp. 2429–2437 (2011)Google Scholar
  27. 27.
    Khosla, A., Xiao, J., Torralba, A., Oliva, A.: Memorability of image regions. In: Advances in Neural Information Processing Systems, pp. 305–313. (2012)Google Scholar
  28. 28.
    Kim, J., Yoon, S., Pavlovic, V.: Relative spatial features for image memorability. In: ACM Multimedia, pp. 761–764 (2013)Google Scholar
  29. 29.
    Cha, M., Kwak, H., Rodriguez, P., Ahn, Y.Y., Moon, S.: Analyzing the video popularity characteristics of large-scale user generated content systems. IEEE/ACM Transactions on Networking (TON) 17(5), 1357–1370 (2009)CrossRefGoogle Scholar
  30. 30.
    Figueiredo, F., Benevenuto, F., Almeida, J.M.: The tube over time: characterizing popularity growth of youtube videos. In: ACM WSDM, pp. 745–754 (2011)Google Scholar
  31. 31.
    Figueiredo, F.: On the prediction of popularity of trends and hits for user generated videos. In: ACM WSDM, pp. 741–746 (2013)Google Scholar
  32. 32.
    Pinto, H., Almeida, J.M., Gonçalves, M.A.: Using early view patterns to predict the popularity of youtube videos. In: ACM WSDM, pp. 365–374 (2013)Google Scholar
  33. 33.
    Redi, M., Merialdo, B.: Where is the beauty?: Retrieving appealing videoscenes by learning flickr-based graded judgments. In: ACM Multimedia, pp. 1363–1364 (2012)Google Scholar
  34. 34.
    Liu, F., Niu, Y., Gleicher, M.: Using web photos for measuring video frame interestingness. In: IJCAI, pp. 2058–2063 (2009)Google Scholar
  35. 35.
    Standing, L., Conezio, J., Haber, R.N.: Perception and memory for pictures: Single-trial learning of 2500 visual stimuli. Psychonomic Science 19(2), 73–74 (1970)CrossRefGoogle Scholar
  36. 36.
    Standing, L.: Learning 10000 pictures. Q. J. Exp. Psychol. 25(2), 207–222 (1973)CrossRefGoogle Scholar
  37. 37.
    Hollingworth, A.: Constructing visual representations of natural scenes: the roles of short-and long-term visual memory. J. Exp. Psychol.: Hum. Percept. Perform. 30(3), 519 (2004)Google Scholar
  38. 38.
    Brady, T.F., Konkle, T., Alvarez, G.A., Oliva, A.: Visual long-term memory has a massive storage capacity for object details. Nat. Acad. Sci. 14325–14329 (2008)Google Scholar
  39. 39.
    Konkle, T., Brady, T.F., Alvarez, G.A., Oliva, A.: Scene memory is more detailed than you think the role of categories in visual long-term memory. Psychological Science 21(11), 1551–1556 (2010)CrossRefGoogle Scholar
  40. 40.
    Konkle, T., Brady, T.F., Alvarez, G.A., Oliva, A.: Conceptual distinctiveness supports detailed visual long-term memory for real-world objects. Journal of Experimental Psychology: General 139(3), 558 (2010)CrossRefGoogle Scholar
  41. 41.
    Khosla, A., Xiao, J., Isola, P., Torralba, A., Oliva, A.: Image memorability and visual inception. In: SIGGRAPH Asia 2012 Technical Briefs, pp. 35. ACM (2012)Google Scholar
  42. 42.
    Khosla, A., Bainbridge, W.A., Torralba, A., Oliva, A.: Modifying the memorability of face photographs. In: ICCV, pp. 3200–3207. IEEE (2013)Google Scholar
  43. 43.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features:spatial pyramid matching for recognizing natural scene categories. In: CVPR, pp. 2169–2178. IEEE (2006)Google Scholar
  44. 44.
    Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. In: IEEE PAMI, pp. 1627–1645 (2010)Google Scholar
  45. 45.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR, vol. 1, pp. 886–893. IEEE (2005)Google Scholar
  46. 46.
    Xiao, J., Hays, J., Ehinger, K.A., Oliva, A., Torralba, A.: Sun database: Large-scale scene recognition from abbey to zoo. In: CVPR, pp. 3485–3492. IEEE (2010)Google Scholar
  47. 47.
    Shechtman, E., Irani, M.: Matching local self-similarities across images and videos. In: CVPR, pp. 1–8. IEEE (2007)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Xesca Amengual
    • 1
    Email author
  • Anna Bosch
    • 1
  • Josep Lluís de la Rosa
    • 1
  1. 1.DEEEA, Centre Easy, Agents Research LABUniversitat de GiroanGironaSpain

Personalised recommendations