Advertisement

Computational Visual Media

, Volume 5, Issue 1, pp 45–58 | Cite as

Discernible image mosaic with edge-aware adaptive tiles

  • Pengfei Xu
  • Jianqiang Ding
  • Hao Zhang
  • Hui HuangEmail author
Open Access
Research Article
  • 25 Downloads

Abstract

We present a novel method to produce discernible image mosaics, with relatively large image tiles replaced by images drawn from a database, to resemble a target image. Compared to existing works on image mosaics, the novelty of our method is two-fold. Firstly, believing that the presence of visual edges in the final image mosaic strongly supports image perception, we develop an edge-aware photo retrieval scheme which emphasizes the preservation of visual edges in the target image. Secondly, unlike most previous works which apply a pre-determined partition to an input image, our image mosaics are composed of adaptive tiles, whose sizes are determined based on the available images in the database and the objective of maximizing resemblance to the target image. We show discernible image mosaics obtained by our method, using image collections of only moderate size. To evaluate our method, we conducted a user study to validate that the image mosaics generated present both globally and locally appropriate visual impressions to the human observers. Visual comparisons with existing techniques demonstrate the superiority of our method in terms of mosaic quality and perceptibility.

Keywords

image mosaic image retrieval image synthesis 

Notes

Acknowledgements

We thank the anonymous reviewers and the editors for their valuable comments. This work was supported in part by the National Natural Science Foundation of China (Nos. 61602310, 61522213, and 61528208), Guangdong Science and Technology Program (No. 2015A030312015), Shenzhen Innovation Program (Nos. JCYJ20170302154106666, KQJSCX20170727101233642), and NSERC (No. 611370).

References

  1. [1]
    Wikipedia-contributors. Photographic mosaic. Available at https://doi.org/en.wikipedia.org/wiki/Photographic_mosaic.
  2. [2]
    Silvers, R.; Hawley, M. Photomosaics. Henry Holt and Co., Inc., 1997.Google Scholar
  3. [3]
    Kim, J.; Pellacini, F. Jigsaw image mosaics. ACM Transactions on Graphics Vol. 21, No. 3, 657–664, 2002.Google Scholar
  4. [4]
    Di Blasi, G.; Gallo, G.; Petralia, M. P. Smart ideas for photomosaic rendering. In: Proceedings of the Eurographics Italian Chapter Conference, Vol. 2006, 267–272, 2006.Google Scholar
  5. [5]
    Orchard, J.; Kaplan, C. S. Cut-out image mosaics. In: Proceedings of the 6th International Symposium on Non-Photorealistic Animation and Rendering, 79–87, 2008.Google Scholar
  6. [6]
    Pavić, D.; Ceumern, U.; Kobbelt, L. GIzMOs: Genuine image mosaics with adaptive tiling. Computer Graphics Forum Vol. 28, No. 8, 2244–2254, 2009.CrossRefGoogle Scholar
  7. [7]
    Miller, J.; Mould, D. Accurate and discernible photocollages. In: Proceedings of the 8th Annual Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging, 115–124, 2012.Google Scholar
  8. [8]
    Zhang, L.; Ma, K.-L.; Yu, J. Adaptively tiled image mosaics utilizing measures of color and region entropy. In: Proceedings of the 9th International Symposium on Visual Information Communication and Interaction, 122–129, 2016.Google Scholar
  9. [9]
    Battiato, S.; Di Blasi, G.; Farinella, G. M.; Gallo, G. Digital mosaic frameworks—An overview. Computer Graphics Forum Vol. 26, No. 4, 794–812, 2007.Google Scholar
  10. [10]
    Dobashi, Y.; Haga, T.; Johan, H.; Nishita, T. A method for creating mosaic images using Voronoi diagrams. In: Proceedings of the Eurographics, Vol. 2, 341–348, 2002.Google Scholar
  11. [11]
    Faustino, G. M.; de Figueiredo, L. H. Simple adaptive mosaic effects. In: Proceedings of the XVIII Brazilian Symposium on Computer Graphics and Image Processing, 315–322, 2005.Google Scholar
  12. [12]
    Zhang, L.; Yu, J. Image mosaics with irregular tiling. In: Proceedings of the 12th International Conference on Computer-Aided Design and Computer Graphics, 155–162, 2011.Google Scholar
  13. [13]
    Lai, Y.-K.; Hu, S.-M.; Martin, R.-R. Surface mosaics. The Visual Computer Vol. 22, Nos. 9–11, 604–611, 2006.CrossRefGoogle Scholar
  14. [14]
    Hausner, A. Simulating decorative mosaics. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, 573–580, 2001.Google Scholar
  15. [15]
    Liu, Y.; Veksler, O.; Juan, O. Generating classic mosaics with graph cuts. Computer Graphics Forum Vol. 29, No. 8, 2387–2399, 2010.CrossRefGoogle Scholar
  16. [16]
    Battiato, S.; Di Blasi, G.; Gallo, G.; Guarnera, G. C.; Puglisi, G. Artificial mosaics by gradient vector flow. In: Proceedings of the Eurographics (Short Papers), 53–56, 2008.Google Scholar
  17. [17]
    Wu, H.; Lyu, X.; Wen, Z. Automatic texture exemplar extraction based on global and local textureness measures. Computational Visual Media Vol. 4, No. 2, 173–184, 2018.CrossRefGoogle Scholar
  18. [18]
    Barnes, C.; Zhang, F. L. A survey of the state-of-the-art in patch-based synthesis. Computational Visual Media Vol. 3, No. 1, 3–20, 2017.CrossRefGoogle Scholar
  19. [19]
    Barnes, C.; Zhang, F.-L.; Lou, L.; Wu, X.; Hu, S.-M. Patchtable: Efficient patch queries for large datasets and applications. ACM Transactions on Graphics Vol. 34, No. 4, Article No. 97, 2015.Google Scholar
  20. [20]
    Di Blasi, G.; Gallo, G.; Petralia, M. Puzzle image mosaic. In: Proceedings of the IASTED/VIIP, 33–37, 2005.Google Scholar
  21. [21]
    Kwan, K. C.; Sinn, L. T.; Han, C.; Wong, T. T.; Fu, C. W. Pyramid of arclength descriptor for generating collage of shapes. ACM Transactions on Graphics Vol. 35, No. 6, Article No. 229, 2016.Google Scholar
  22. [22]
    Huang, H.; Zhang, L.; Zhang, H.-C. Arcimboldo-like collage using internet images. ACM Transactions on Graphics Vol. 30, No. 6, Article No. 155, 2011.Google Scholar
  23. [23]
    Reinert, B.; Ritschel, T.; Seidel, H.-P. Interactive by-example design of artistic packing layouts. ACM Transactions on Graphics Vol. 32, No. 6, Article No. 218, 2013.Google Scholar
  24. [24]
    Zou, C.; Cao, J.; Ranaweera, W.; Alhashim, I.; Tan, P.; Sheffer, A.; Zhang, H. Legible compact calligrams. ACM Transactions on Graphics Vol. 35, No. 4, Article No. 122, 2016.Google Scholar
  25. [25]
    Hu, S.-M.; Zhang, F.-L.; Wang, M.; Martin, R. R.; Wang, J. PatchNet: A patch-based image representation for interactive library-driven image editing. ACM Transactions on Graphics Vol. 32, No. 6, Article No. 196, 2013.Google Scholar
  26. [26]
    Zhang, F.-L.; Wang, J.; Shechtman, E.; Zhou, Z.-Y.; Shi, J.-X.; Hu, S.-M. PlenoPatch: Patch-based plenoptic image manipulation. IEEE Transactions on Visualization and Computer Graphics Vol. 23, No. 5, 1561–1573, 2017.CrossRefGoogle Scholar
  27. [27]
    Rother, C.; Bordeaux, L.; Hamadi, Y.; Blake, A. AutoCollage. ACM Transactions on Graphics Vol. 25, No. 3, 847–852, 2006.CrossRefGoogle Scholar
  28. [28]
    Yu, Z.; Lu, L.; Guo, Y.; Fan, R.; Liu, M.; Wang, W. Content-aware photo collage using circle packing. IEEE Transactions on Visualization and Computer Graphics Vol. 20, No. 2, 182–195, 2014.CrossRefGoogle Scholar
  29. [29]
    Goferman, S.; Tal, A.; Zelnik-Manor, L. Puzzle-like collage. Computer Graphics Forum Vol. 29, No. 2, 459–468, 2010.CrossRefGoogle Scholar
  30. [30]
    Oliva, A.; Torralba, A.; Schyns, P. G. Hybrid images. ACM Transactions on Graphics Vol. 25, No. 3, 527–532, 2006.CrossRefGoogle Scholar
  31. [31]
    Chu, H.-K.; Hsu, W.-H.; Mitra, N. J.; Cohen-Or, D.; Wong, T.-T.; Lee, T.-Y. Camouflage images. ACM Transactions on Graphics Vol. 29, No. 4, Article No. 51, 2010.Google Scholar
  32. [32]
    Tong, Q.; Zhang, S.-H.; Hu, S.-M.; Martin, R. R. Hidden images. In: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Non-Photorealistic Animation and Rendering, 27–34, 2011.Google Scholar
  33. [33]
    Farbman, Z.; Fattal, R.; Lischinski, D.; Szeliski, R. Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Transactions on Graphics Vol. 27, No. 3, Article No. 67, 2008.Google Scholar
  34. [34]
    Muja, M.; Lowe, D. G. Flann: Fast library for approximate nearest neighbors. 2009. Available at https://doi.org/www.cs.ubc.ca/research/ann/.Google Scholar
  35. [35]
    Dalal, N.; Triggs, B. Histograms of oriented gradients for human detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, 886–893, 2005.Google Scholar
  36. [36]
    Zitnick, C. L.; Dollár, P. Edge boxes: Locating object proposals from edges. In: Computer Vision — ECCV 2014. Lecture Notes in Computer Science, Vol. 8693. Fleet, D.; Pajdla, T.; Schiele, B.; Tuytelaars, T. Eds. Springer Cham, 391–405, 2014.Google Scholar
  37. [37]
    Zhou, B.; Lapedriza, A.; Khosla, A.; Oliva, A.; Torralba, A. Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 40, No. 6, 1452–1464, 2018.CrossRefGoogle Scholar
  38. [38]
    Rapid-Mosaic. Foto-mosaic-edda. Available at https://doi.org/fmedda.com/en/home.

Copyright information

© The Author(s) 2019

This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit https://doi.org/creativecommons.org/licenses/by/4.0/.

Other papers from this open access journal are available free of charge from https://doi.org/www.springer.com/journal/41095. To submit a manuscript, please go to https://doi.org/www.editorialmanager.com/cvmj.

Authors and Affiliations

  • Pengfei Xu
    • 1
  • Jianqiang Ding
    • 1
  • Hao Zhang
    • 2
  • Hui Huang
    • 1
    Email author
  1. 1.College of Computer Science and Software EngineeringShenzhen UniversityShenzhen, GuangdongChina
  2. 2.School of Computing ScienceSimon Fraser UniversityVancouverCanada

Personalised recommendations