Saliency Based Image Cropping

  • Edoardo Ardizzone
  • Alessandro Bruno
  • Giuseppe Mazzola
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8156)

Abstract

Image cropping is a technique that is used to select the most relevant areas of an image, discarding the useless ones. Handmade selection, especially in case of large photo collections, is a time consuming task. Automatic image cropping techniques may help users, suggesting to them which part of the image is the most relevant, according to specific criteria. We suppose that the most visually salient areas of a photo are also the most relevant ones to the users. In this paper we present an extended version of our previously proposed method, to extract the saliency map of an image, which is based on the analysis of the distribution of the interest points of the image. Three different interest point extraction algorithms are evaluated within an automatic image cropping system, to study the effectiveness of the related saliency maps for this task. We furthermore compared our results with two state of the art saliency detection techniques. Tests have been conducted onto an online available dataset, made of 5000 images which have been manually labeled by 9 users.

Keywords

Image Cropping Visual Saliency Visual Perception Saliency Map 

References

  1. 1.
    Ardizzone, E., Bruno, A., Mazzola, G.: Visual saliency by keypoints distribution analysis. In: Maino, G., Foresti, G.L. (eds.) ICIAP 2011, Part I. LNCS, vol. 6978, pp. 691–699. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  2. 2.
    Suh, B., Ling, H., Bederson, B.B., Jacobs, D.W.: Automatic Thumbnail Cropping and its Effectiveness. In: Proc. of the 16th ACM Symposium on User Interface Software and Technology, pp. 95–104 (2003)Google Scholar
  3. 3.
    Ma, M., Guo, J.K.: Automatic Image Cropping for Mobile Devices with Built-in Camera. In: Proc. of the Consumer Communication & Networking Conf., pp. 710–711 (January 2004)Google Scholar
  4. 4.
    Zhang, M., Zhang, L., Sun, Y., Feng, L., Ma, W.: Auto Cropping for Digital Photographs. In: IEEE International Conference on Multimedia and Expo (2005)Google Scholar
  5. 5.
    Santella, A., Agrawala, M., DeCarlo, D., Salesin, D., Cohen, M.F.: Gaze-based interaction for semi-automatic photo cropping. In: CHI 2006, pp. 771–780 (2006)Google Scholar
  6. 6.
    Stentiford, F.: Attention Based Auto Image Cropping. In: ICVS Workshop on Computational Attention & Applications (2007)Google Scholar
  7. 7.
    Ciocca, G., Cusano, C., Gasparini, F., Schettini, R.: Self-Adaptive Image Cropping for Small Displays. IEEE Trans. on Cons. Electronics 53(4), 1622–1627 (2007)CrossRefGoogle Scholar
  8. 8.
    She, J., Wang, D., Song, M.: Automatic image cropping using sparse coding. In: 2011 First Asian Conference on Pattern Recognition (ACPR), pp. 490–494 (2011)Google Scholar
  9. 9.
    Luo, J.: Subject Content-Based Intelligent Cropping of Digital Photos. In: IEEE International Conference on Multimedia and Expo, pp. 2218–2221 (2007)Google Scholar
  10. 10.
    Nishiyama, M., Okabe, T., Sato, Y., Sato, I.: Sensation-based photo cropping. In: Proceedings of the 17th International Conference on Multimedia 2009, pp. 669–672. ACM, Vancouver (2009)Google Scholar
  11. 11.
    Cheng, B., Ni, B., Yan, S., Tian, Q.: Learning to photograph. In: Proceedings of the International Conference on Multimedia, Ser. MM 2010, pp. 291–300. ACM (2010)Google Scholar
  12. 12.
    Bhattacharya, S., Sukthankar, R., Shah, M.: A framework for photo-quality assessment and enhancement based on visual aesthetics. In: Proceedings of the International Conference on Multimedia (MM 2010), pp. 271–280. ACM, New York (2010)Google Scholar
  13. 13.
    Liu, L., Chen, R., Wolf, L., Cohen-Or, D.: Optimizing photo composition. Computer Graphic Forum 29(2), 469–478 (2010)CrossRefGoogle Scholar
  14. 14.
    Ahn, S., Agrawala, M., Hartmann, B., Barsky, B.A.: Image Cropping: Collection and Analysis of Crowdsourced Data Technical Report No. UCB/EECS-2012-94 (2012)Google Scholar
  15. 15.
    McManus, I.C., Zhou, F.A., l’Anson, S., Waterfield, L., Stöver, K., Cook, R.: The psychometrics of photographic cropping:The influence of colour, meaning, and expertise. Perception 40(3), 332–357 (2011)Google Scholar
  16. 16.
    Frintrop, S., Rome, E., Christensen, H.I.: Computational visual attention systems and their cognitive foundations: A survey. ACM Transactions on Applied Perception (TAP) 7(1), Article 6 (2010)Google Scholar
  17. 17.
    Judd, T., Ehinger, K., Durand, F., Torralba, A.: Learning to predict where humans look. In: 12th International Conference on Computer Vision (2009)Google Scholar
  18. 18.
    Achanta, R., Hemami, S., Estrada, F., Süsstrunk, S.: Frequency-tuned Salient Region Detection. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR 2009), pp. 1597–1604 (2009)Google Scholar
  19. 19.
    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(11), 1254 (1998)CrossRefGoogle Scholar
  20. 20.
    Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. Advances in neural information processing systems, vol. 19, pp. 545–552. MIT Press, Cambridge (2007)Google Scholar
  21. 21.
    Lowe, D.G.: Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  22. 22.
    Bay, H., Tuytelaars, T., Van Gool, L.: SURF: Speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part I. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  23. 23.
    Harris, C., Stephens, M.: A combined edge corner detector. In: 4th Alvey Vision Conference (1998)Google Scholar
  24. 24.
    Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., Shum, H.Y.: Learning to detect a salient object. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(2), 353–367 (2011)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Edoardo Ardizzone
    • 1
  • Alessandro Bruno
    • 1
  • Giuseppe Mazzola
    • 1
  1. 1.Dipartimento di Ingegneria Chimica, Gestionale, Informatica, MeccanicaUniversità degli studi di PalermoItaly

Personalised recommendations