Rating Image Aesthetics Using a Crowd Sourcing Approach

  • Abhishek Agrawal
  • Vittal Premachandran
  • Ramakrishna Kakarala
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8334)


Any system that is able to reliably measure the aesthetic appeal of photographs would be of considerable importance to the digital imaging industry. Researchers have built automated rating systems using machine learning techniques applied to features extracted from images. In this paper, we study the effectiveness of ACQUINE, a comprehensive and publicly available rating system, using data obtained from voters in a crowd sourced manner. We analyze the effect of voting using a simple binary like/dislike rating in comparison to a numerical 10 point scale. We also show that global measures of image quality, such as contrast or colorfulness, do not correlate well with human ratings. The role of composition in determining human rating of aesthetics is discussed.


Image quality Image color analysis 


  1. 1.
    Datta, R., Zewang, J.: ACQUINE: aesthetic quality inference engine - real time automatic ratings of photo aesthetics. In: Multimedia Information Retrieval, pp. 421–424 (2010)Google Scholar
  2. 2.
    Murray, N., Marchesotti, L., Perronnin, F.: Ava: A large-scale database for aesthetic visual analysis. In: Computer Vision and Pattern Recognition (CVPR), pp. 2408–2415 (2012)Google Scholar
  3. 3.
    Sachs, T.S., Kakarala, R., Castleman, S.L., Rajan, D.: A data-driven approach to understanding skill in photographic composition. In: Koch, R., Huang, F. (eds.) ACCV 2010 Workshops, Part II. LNCS, vol. 6469, pp. 112–121. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  4. 4.
    Kakarala, R., Sachs, T.S., Premachandran, V.: Comparing automated and human ratings of photographic aesthetics. In: Proceedings of the 19th Color Imaging Conference, San Jose, CA (2011)Google Scholar
  5. 5.
    Savakis, A., Etz, S., Loui, A.: Evaluation of image appeal in consumer photography. In: SPIE Human Vision and Electronic Imaging (2000)Google Scholar
  6. 6.
    Ke, Y., Tang, X., Jing, F.: The design of high-level features for photo quality assessment. In: Computer Vision and Pattern Recognition, pp. 419–426 (2006)Google Scholar
  7. 7.
    Tong, H., Li, M., Zhang, H.-J., He, J., Zhang, C.: Classification of digital photos taken by photographers or home users. In: Aizawa, K., Nakamura, Y., Satoh, S. (eds.) PCM 2004. LNCS, vol. 3331, pp. 198–205. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  8. 8.
    Datta, R., Li, J., Wang, J.Z.: Algorithmic inferencing of aesthetics and emotion in natural images: An exposition. In: International Conference on Image Processing, pp. 105–108 (2008)Google Scholar
  9. 9.
    Datta, R., Joshi, D., Li, J., Wang, J.Z.: Studying aesthetics in photographic images using a computational approach. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 288–301. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  10. 10.
    Marchesotti, L., Perronnin, F., Larlus, D., Csurka, G.: Assessing the aesthetic quality of photographs using generic image descriptors. In: International Conference on Computer Vision, pp. 1784–1791 (2011)Google Scholar
  11. 11.
    Bargagliotti, A.E., Li, Y.: Decision making using rating systems. When scale meets binary. MPRA Paper 16947. University Library of Munich, Germany (2006)Google Scholar
  12. 12.
    Cosely, D., Lam, S.K., Istvan, A., Konstan, J.A., Riedl, J.: Is seeing believing? How recommender system interfaces affect users’ opinion. In: CHI, pp. 585–592 (2003)Google Scholar
  13. 13.
    Rubner, R., Tomasi, C., Guibas, L.J.: The earth mover’s distance as a metric for image retrieval. International Journal of Computer Vision, 99–121 (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Abhishek Agrawal
    • 1
  • Vittal Premachandran
    • 1
  • Ramakrishna Kakarala
    • 1
  1. 1.School of Computer EngineeringNanyang Technological UniversitySingaporeSingapore

Personalised recommendations