Skip to main content
SpringerLink
Log in

Style-based exploration of illustration datasets

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Searching by style in illustration data sets is a particular problem in Information Retrieval which has received little attention so far. One of its main problems is that the perception of style is highly subjective, which makes labeling styles a very difficult task. Despite being difficult to predict computationally, certain properties such as colorfulness, line style or shading can be successfully captured by existing style metrics. However, there is little knowledge about how we distinguish between different styles and how these metrics can be used to guide users in style-based interactions. In this paper, we propose several contributions towards a better comprehension of illustration style and its usefulness for data exploration and retrieval. First, we provide new insights about how we perceive style in illustration. Second, we evaluate a handmade style clustering of clip art pieces with an existing style metric to analyze how this metric aligns with expert knowledge. Finally, we propose a method for efficient navigation and exploration of large clip art data sets which takes into account both semantic labeling of the data and its style. Our approach combines hierarchical clustering with dimensionality reduction techniques, and strategic sampling to obtain intuitive visualizations and useful visualizations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. Note that candidate images in a single node become hidden images after step 1.

References

  1. Averkiou M, Kim V, Zheng Y, Mitra NJ (2014) Shapesynth: parameterizing model collections for coupled shape exploration and synthesis. Computer Graphics Forum (Proc. Eurographics)

  2. Bae S, Paris S, Durand F (2006) Two-scale tone management for photographic look. ACM Trans Graphics 25(3)

  3. Campbell NDF, Kautz J (2014) Learning a manifold of fonts. ACM Trans Graphics (Proc SIGGRAPH) 33(4)

  4. Durand F (2002) An invitation to discuss computer depiction. In: Proceedings of NPAR

  5. Eitz M, Hay J, Alexa M (2012) How Do Humans Sketch Objects? ACM Trans Graphics (Proc SIGGRAPH)

  6. Fried O, DiVerdi S, Halber M, Sizikova E, Finkelstein A (2015) IsoMatch: Creating informative grid layouts. Computer Graphics Forum (Proc Eurographics) 34 (2)

  7. Frisby JP, Stone JV (2010) Seeing: the computational approach to biological vision. MIT Press

  8. Garces E, Agarwala A, Gutierrez D, Hertzmann A (2014) A similarity measure for illustration style. ACM Trans Graphics (Proc SIGGRAPH) 33(4)

  9. Han Z, Liu Z, Han J, Bu S (2014) 3d shape creation by style transfer. Vis Comput 31(9)

  10. Hertzmann A, Jacobs CE, Oliver N, Curless B, Salesin DH (2001) Image analogies. In: Proceedings of SIGGRAPH

  11. Huang SS, Shamir A, Shen CH, Zhang H, Sheffer A, Hu SM, Cohen-Or D (2013) Qualitative organization of collections of shapes via quartet analysis. ACM Trans Graphics (Proc SIGGRAPH) 32(4)

  12. Jin R, Wang S, Zhou Y (2009) Regularized distance metric learning: theory and algorithm. In: Proceedings of neural information processing systems

  13. Kalogerakis E, Nowrouzezahrai D, Breslav S, Hertzmann A (2012) Learning hatching for Pen-and-Ink illustration of surfaces. ACM Trans Graphics 31

  14. Kleiman Y, Fish N, Lanir J, Cohen-Or D (2013) Dynamic maps for exploring and browsing shapes. In: Proceedings Eurographics/ACMSIGGRAPH symposium on geometry processing

  15. Kohonen T (1990) The self-organizing map. Proc IEEE 78(9)

  16. Kulis B (2013) Metric learning: a survey. Foundations and Trends in Machine Learning 5(4)

  17. Li H, Zhang H, Wang Y, Cao J, Shamir A, Cohen-Or D (2013) Curve style analysis in a set of shapes. Comput Graph Forum 32(6):77–88

    Article  Google Scholar 

  18. Liu T, Hertzmann A, Li W, Funkhouser T (2015) Style compatibility for 3d furniture models. ACM Trans Graphics (Proc SIGGRAPH) 34(4)

  19. Lun Z, Kalogerakis E, Sheffer A (2015) Elements of style: learning perceptual shape style similarity. ACM Trans Graphics (Proc SIGGRAPH) 34(4)

  20. Luo Y, Liu T, Tao D, Xu C (2014) Decomposition-based transfer distance metric learning for image classification. IEEE Trans Image Process 23(9)

  21. McFee B, Lanckriet G (2010) Metric learning to rank. In: Proceedings of international conference on machine learning

  22. Nguyen CH, Ritschel T, Seidel HP (2015) Data-driven color manifolds. ACM Trans Graphics 34(2)

  23. O’Donovan P, Agarwala A, Hertzmann A (2014) Collaborative filtering of color aesthetics. In: Proceedings Computational Aesthetics

  24. O’Donovan P, Lı̄beks J, Agarwala A, Hertzmann A (2014) Exploratory font selection using crowdsourced attributes. ACM Trans Graphics (Proc SIGGRAPH) 33

  25. Parikh D, Grauman K (2011) Relative attributes. In: Proceedings of IEEE international conference on computer vision

  26. Reinert B, Ritschel T, Seidel HP (2013) Interactive by-example design of artistic packing layouts. ACM Trans Graphics (Proc SIGGRAPH Asia) 31(6)

  27. Rubinstein M, Gutierrez D, Sorkine O, Shamir A (2010) A comparative study of image retargeting. ACM Trans Graphics (Proc SIGGRAPH Asia) 29(5)

  28. Saleh B, Dontcheva M, Hertzmann A, Liu Z (2015) Learning style similarity for searching infographics. In: Proceedings graphics interface conference

  29. Schultz M, Joachims T (2003) Learning a distance metric from relative comparisons. In: Proceedings of neural information processing systems

  30. Shapira L, Shamir A, Cohen-Or D (2009) Image appearance exploration by model-based navigation. Comput Graph Forum (Proc Eurographics) 28(2)

  31. Sidi O, van Kaick O, Kleiman Y, Zhang H, Cohen-Or D (2011) Unsupervised co-segmentation of a set of shapes via descriptor-space spectral clustering. ACM Trans Graphics (Proc SIGGRAPH Asia) 30(6)

  32. Tenenbaum JB, Freeman WT (2000) Separating style and content with bilinear models. Neural Comput 12(6)

  33. Ward JH (1963) Hierarchical grouping to optimize an objective function. J Am Stat Assoc 58(301)

  34. Willats J, Durand F (2005) Defining pictorial style: lessons from linguistics and computer graphics. Axiomathes 15(3)

  35. Yamaguchi K, Kiapour MH, Ortiz LE, Berg TL (2015) Retrieving similar styles to parse clothing. IEEE Trans on Pattern Anal Machine Intelligence 37(5)

  36. Yang L (2006) Distance metric learning: a comprehensive survey. Tech Rep

Download references

Acknowledgments

We would like to thank all reviewers for their thoughtful comments. We also thank Carlos Bobed for insightful comments and proofreading the paper. This work was partially supported by the the Gobierno de Aragon, the Ministerio de Economia y Competitividad (project LIGHTSLICE and BLINK), and a generous gift from Adobe Systems.

Author information

Authors and Affiliations

  1. IIS Department, University of Zaragoza, Zaragoza, Spain

    Elena Garces & Diego Gutierrez

  2. Google Research, Seattle, WA, USA

    Aseem Agarwala

  3. Adobe Research, San Francisco, CA, USA

    Aaron Hertzmann

Authors
  1. Elena Garces
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aseem Agarwala
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aaron Hertzmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Diego Gutierrez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Elena Garces.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(WMV 12.7 MB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Garces, E., Agarwala, A., Hertzmann, A. et al. Style-based exploration of illustration datasets. Multimed Tools Appl 76, 13067–13086 (2017). https://doi.org/10.1007/s11042-016-3702-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3702-x

Keywords

Search

Navigation