Towards Computational Models of the Visual Aesthetic Appeal of Consumer Videos

  • Anush K. Moorthy
  • Pere Obrador
  • Nuria Oliver
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6315)


In this paper, we tackle the problem of characterizing the aesthetic appeal of consumer videos and automatically classifying them into high or low aesthetic appeal. First, we conduct a controlled user study to collect ratings on the aesthetic value of 160 consumer videos. Next, we propose and evaluate a set of low level features that are combined in a hierarchical way in order to model the aesthetic appeal of consumer videos. After selecting the 7 most discriminative features, we successfully classify aesthetically appealing vs. aesthetically unappealing videos with a 73% classification accuracy using a support vector machine.


Support Vector Machine Ground Truth Motion Vector Mean Opinion Score Image Quality Assessment 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Junee, R.: 20 Hours of Video Uploaded Every Minute! (2009),
  2. 2.
    Verna, P.: A Spotlight on UGC Participants (2009),
  3. 3.
  4. 4.
    Messaris, P.: Visual persuasion: the role of images in advertising. Sage Publications Inc., Thousand Oaks (1997)Google Scholar
  5. 5.
    Wang, Z., Sheikh, H.R., Bovik, A.C.: Objective video quality assessment. In: The Handbook of Video Databases: Design and Applications, pp. 1041–1078 (2003)Google Scholar
  6. 6.
    Datta, R., Joshi, D., Li, J., Wang, J.: Studying aesthetics in photographic images using a computational approach. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 288–301. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  7. 7.
    Ke, Y., Tang, X., Jing, F.: The design of high-level features for photo quality assessment. In: IEEE Conf. Comp. Vis. Pat. Recog., vol. 1 (2006)Google Scholar
  8. 8.
    Luo, Y., Tang, X.: Photo and video quality evaluation: Focusing on the subject. In: Eur. Conf. Comp. Vis., pp. 386–399 (2008)Google Scholar
  9. 9.
    Obrador, P.: Region based image appeal metric for consumer photos. In: IEEE Work. Mult. Sig. Proc., pp. 696–701 (2008)Google Scholar
  10. 10.
    Tong, H., Li, M., Zhang, H., He, J., Zhang, C.: Classification of digital photos taken by photographers or home users. LNCS, pp. 198–205. Springer, Heidelberg (2004)Google Scholar
  11. 11.
    Moorthy, A.K., Bovik, A.C.: Visual importance pooling for image quality assessment. IEEE Jnl. Sel. Top. Sig. Proc. 3(2), 193–201 (2009)CrossRefGoogle Scholar
  12. 12.
    Savakis, A.E., Etz, S.P., Loui, A.C.: Evaluation of image appeal in consumer photography. In: SPIE Proc., Human Vis. Elec. Img., pp. 111–121 (2000)Google Scholar
  13. 13.
    Datta, R., Li, J., Wang, J.Z.: Algorithmic Inferencing of Aesthetics and Emotion in Natural Images: An Exposition. In: IEEE Intl. Conf. Image Proc., pp. 105–108 (2008)Google Scholar
  14. 14.
    Amatriain, X., Pujol, J.M., Oliver, N.: I Like It, I Like It Not. In: Houben, G.-J., McCalla, G., Pianesi, F., Zancanaro, M. (eds.) UMAP 2009. LNCS, vol. 5535, pp. 247–258. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  15. 15.
    Pinson, M.H., Wolf, S.: Comparing subjective video quality testing methodologies. In: Vis. Comm. and Imag., SPIE, vol. 5150, pp. 573–582 (2003)Google Scholar
  16. 16.
    Beerends, J.G., De Caluwe, F.: The influence of video quality on perceived audio quality and vice versa. Jnl. Aud. Engg. Soc. 47, 355–362 (1999)Google Scholar
  17. 17.
    van Dijk, A.M., Martens, J.B., Watson, A.B.: Quality asessment of coded images using numerical category scaling. In: SPIE Adv. Image Video Comm. Storage Tech., vol. 2451, pp. 90–101 (1995)Google Scholar
  18. 18.
    BT.: 500-11: Methodology for the subjective assessment of the quality of television pictures. In: International Telecommuncation Union, Geneva, Switzerland (2002)Google Scholar
  19. 19.
    LIVE video quality assessment databases,
  20. 20.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: From error visibility to structural similarity. IEEE Tran. Image Proc. 13, 600–612 (2004)CrossRefGoogle Scholar
  21. 21.
    Yan, W., Kankanhalli, M.: Detection and removal of lighting & shaking artifacts in home videos. In: Proc. ACM Conf. Mult., pp. 107–116 (2002)Google Scholar
  22. 22.
    Dai, Z., Wu, Y.: Where Are Focused Places of a Photo? In: Qiu, G., Leung, C., Xue, X.-Y., Laurini, R. (eds.) VISUAL 2007. LNCS, vol. 4781, pp. 73–83. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  23. 23.
    Hasler, D., Susstrunk, S., GmbH, L., Steinfurt, G.: Measuring colourfulness in natural images. In: SPIE/IS&T Hum. Vis. Elec. Img., vol. 5007, pp. 87–95 (2003)Google Scholar
  24. 24.
    Cohen-Or, D., Sorkine, O., Gal, R., Leyvand, T., Xu, Y.: Color harmonization. In: Proc. ACM SIGGRAPH, vol. 25, pp. 624–630 (2006)Google Scholar
  25. 25.
    Li, C., Chen, T.: Aesthetic Visual Quality Assessment of Paintings. IEEE Jnl. Sel. Top. Sig. Proc. 3, 236–252 (2009)CrossRefGoogle Scholar
  26. 26.
    Wang, Z., Sheikh, H.R., Bovik, A.C.: No-reference perceptual quality assessment of JPEG compressed images. In: IEEE Intl. Conf. Image Proc., vol. 1, pp. 477–480 (2002)Google Scholar
  27. 27.
    Richardson, I.: H. 264 and MPEG-4 video compression: video coding for next-generation multimedia. John Wiley & Sons Inc, Chichester (2003)CrossRefGoogle Scholar
  28. 28.
    Freeman, M.: The image. revised edn. William Collins Sons & Co. Ltd. (1990)Google Scholar
  29. 29.
    Vapnik, V.: The Nature of Statistical Learning Theory. Springer, Heidelberg (2000)zbMATHGoogle Scholar
  30. 30.
    Chang, C., Lin, C.: LIBSVM: a library for support vector machines (2001),

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Anush K. Moorthy
    • 1
  • Pere Obrador
    • 1
  • Nuria Oliver
    • 1
  1. 1.Telefonica ResearchBarcelonaSpain

Personalised recommendations