Automatic Image Cropping and Selection Using Saliency: An Application to Historical Manuscripts

  • Marcella Cornia
  • Stefano Pini
  • Lorenzo Baraldi
  • Rita Cucchiara
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 806)

Abstract

Automatic image cropping techniques are particularly important to improve the visual quality of cropped images and can be applied to a wide range of applications such as photo-editing, image compression, and thumbnail selection. In this paper, we propose a saliency-based image cropping method which produces significant cropped images by only relying on the corresponding saliency maps. Experiments on standard image cropping datasets demonstrate the benefit of the proposed solution with respect to other cropping methods. Moreover, we present an image selection method that can be effectively applied to automatically select the most representative pages of historical manuscripts thus improving the navigation of historical digital libraries.

Keywords

Image cropping Image selection Saliency Digital libraries 

Notes

Acknowledgment

We gratefully acknowledge the Estense Gallery of Modena for the availability of the digitized historical manuscripts used in this work. We also acknowledge the CINECA award under the ISCRA initiative, for the availability of high performance computing resources and support.

References

  1. 1.
    Avidan, S., Shamir, A.: Seam carving for content-aware image resizing. ACM Trans. Graph. 26(3), 10 (2007)CrossRefGoogle Scholar
  2. 2.
    Balducci, F., Grana, C.: Affective classification of gaming activities coming from RPG gaming sessions. In: Tian, F., Gatzidis, C., El Rhalibi, A., Tang, W., Charles, F. (eds.) Edutainment 2017. LNCS, vol. 10345, pp. 93–100. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-65849-0_11 CrossRefGoogle Scholar
  3. 3.
    Bhattacharya, S., Sukthankar, R., Shah, M.: A framework for photo-quality assessment and enhancement based on visual aesthetics. In: ACM International Conference on Multimedia (2010)Google Scholar
  4. 4.
    Bolelli, F.: Indexing of historical document images: ad hoc dewarping technique for handwritten text. In: Grana, C., Baraldi, L. (eds.) IRCDL 2017. CCIS, vol. 733, pp. 45–55. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-68130-6_4 CrossRefGoogle Scholar
  5. 5.
    Chen, J., Bai, G., Liang, S., Li, Z.: Automatic image cropping: a computational complexity study. In: IEEE International Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  6. 6.
    Chen, Y.L., Huang, T.W., Chang, K.H., Tsai, Y.C., Chen, H.T., Chen, B.Y.: Quantitative analysis of automatic image cropping algorithms: a dataset and comparative study. In: Winter Conference on Applications of Computer Vision (2017)Google Scholar
  7. 7.
    Chen, Y.L., Klopp, J., Sun, M., Chien, S.Y., Ma, K.L.: Learning to compose with professional photographs on the web. arXiv preprint arXiv:1702.00503 (2017)
  8. 8.
    Cheng, B., Ni, B., Yan, S., Tian, Q.: Learning to photograph. In: ACM International Conference on Multimedia (2010)Google Scholar
  9. 9.
    Ciocca, G., Cusano, C., Gasparini, F., Schettini, R.: Self-adaptive image cropping for small displays. IEEE Trans. Consum. Electron. 53(4), 1622–1627 (2007)CrossRefGoogle Scholar
  10. 10.
    Cornia, M., Baraldi, L., Serra, G., Cucchiara, R.: A deep multi-level network for saliency prediction. In: International Conference on Pattern Recognition (2016)Google Scholar
  11. 11.
    Cornia, M., Baraldi, L., Serra, G., Cucchiara, R.: Multi-level net: a visual saliency prediction model. In: European Conference on Computer Vision Workshops (2016)Google Scholar
  12. 12.
    Cornia, M., Baraldi, L., Serra, G., Cucchiara, R.: Predicting human eye fixations via an LSTM-based saliency attentive model. arXiv preprint arXiv:1611.09571 (2017)
  13. 13.
    Cucchiara, R., Grana, C., Prati, A.: Semantic transcoding for live video server. In: ACM International Conference on Multimedia (2002)Google Scholar
  14. 14.
    Kang, H.W., Hua, X.S.: To learn representativeness of video frames. In: ACM International Conference on Multimedia (2005)Google Scholar
  15. 15.
    Ke, Y., Tang, X., Jing, F.: The design of high-level features for photo quality assessment. In: IEEE International Conference on Computer Vision and Pattern Recognition (2006)Google Scholar
  16. 16.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  17. 17.
    Li, D., Wu, H., Zhang, J., Huang, K.: A2-RL: aesthetics aware reinforcement learning for automatic image cropping. arXiv preprint arXiv:1709.04595 (2017)
  18. 18.
    Liu, C., Huang, Q., Jiang, S.: Query sensitive dynamic web video thumbnail generation. In: IEEE International Conference on Image Processing (2011)Google Scholar
  19. 19.
    Liu, W., Mei, T., Zhang, Y., Che, C., Luo, J.: Multi-task deep visual-semantic embedding for video thumbnail selection. In: IEEE International Conference on Computer Vision and Pattern Recognition (2015)Google Scholar
  20. 20.
    Luo, J., Papin, C., Costello, K.: Towards extracting semantically meaningful key frames from personal video clips: from humans to computers. IEEE Trans. Circ. Syst. Video Technol. 19(2), 289–301 (2009)CrossRefGoogle Scholar
  21. 21.
    Ma, M., Guo, J.K.: Automatic image cropping for mobile device with built-in camera. In: Consumer Communications and Networking Conference (2004)Google Scholar
  22. 22.
    Nishiyama, M., Okabe, T., Sato, Y., Sato, I.: Sensation-based photo cropping. In: ACM International Conference on Multimedia (2009)Google Scholar
  23. 23.
    Park, J., Lee, J.Y., Tai, Y.W., Kweon, I.S.: Modeling photo composition and its application to photo re-arrangement. In: IEEE International Conference on Image Processing (2012)Google Scholar
  24. 24.
    Santella, A., Agrawala, M., DeCarlo, D., Salesin, D., Cohen, M.: Gaze-based interaction for semi-automatic photo cropping. In: SIGCHI Conference on Human Factors in Computing Systems (2006)Google Scholar
  25. 25.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  26. 26.
    Stentiford, F.: Attention based auto image cropping. In: Workshop on Computational Attention and Applications, ICVS (2007)Google Scholar
  27. 27.
    Suh, B., Ling, H., Bederson, B.B., Jacobs, D.W.: Automatic thumbnail cropping and its effectiveness. In: ACM Symposium on User Interface Software and Technology (2003)Google Scholar
  28. 28.
    Tang, X., Luo, W., Wang, X.: Content-based photo quality assessment. IEEE Trans. Multimed. 15(8), 1930–1943 (2013)CrossRefGoogle Scholar
  29. 29.
    Wang, M., Hong, R., Li, G., Zha, Z.J., Yan, S., Chua, T.S.: Event driven web video summarization by tag localization and key-shot identification. IEEE Trans. Multimed. 14(4), 975–985 (2012)CrossRefGoogle Scholar
  30. 30.
    Yan, J., Lin, S., Bing Kang, S., Tang, X.: Learning the change for automatic image cropping. In: IEEE International Conference on Computer Vision and Pattern Recognition (2013)Google Scholar
  31. 31.
    Zhang, L., Song, M., Zhao, Q., Liu, X., Bu, J., Chen, C.: Probabilistic graphlet transfer for photo cropping. IEEE Trans. Image Process. 22(2), 802–815 (2013)MathSciNetCrossRefMATHGoogle Scholar
  32. 32.
    Zhang, M., Zhang, L., Sun, Y., Feng, L., Ma, W.: Auto cropping for digital photographs. In: ICME (2005)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Marcella Cornia
    • 1
  • Stefano Pini
    • 1
  • Lorenzo Baraldi
    • 1
  • Rita Cucchiara
    • 1
  1. 1.University of Modena and Reggio EmiliaModenaItaly

Personalised recommendations