Optimal Patch Assignment for Statistically Constrained Texture Synthesis

  • Jorge GutierrezEmail author
  • Julien Rabin
  • Bruno Galerne
  • Thomas Hurtut
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10302)


This article introduces a new model for patch-based texture synthesis that controls the distribution of patches in the synthesized texture. The proposed approach relies on an optimal assignment of patches over decimated pixel grids. This assignment problem formulates the synthesis as the minimization of a discrepancy measure between input’s and output’s patches through their optimal permutation. The resulting non-convex optimization problem is addressed with an iterative algorithm alternating between a patch assignment step and a patch aggregation step. We show that this model statistically constrains the output texture content, while inheriting the structure-preserving property of patch-based methods. We also propose a relaxed patch assignment extension that increases the robustness to non-stationnary textures.


Example-based texture synthesis Patch matching Optimal assignment 


  1. 1.
    Aguerrebere, C., Gousseau, Y., Tartavel, G.: Exemplar-based texture synthesis: the Efros-Leung algorithm. IPOL 3, 223–241 (2013)CrossRefGoogle Scholar
  2. 2.
    Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.B., Patchmatch: A randomized correspondence algorithm for structural image editing. In: SIGGRAPH, pp. 24:1–24:11. ACM (2009)Google Scholar
  3. 3.
    Bougleux, S., Brun, L.: Linear sum assignment with edition. preprint, arXiv:1603.04380 (2016)
  4. 4.
    Burkard, R., Dell’Amico, M., Martello, S.: Assignment Problems. Society for Industrial and Applied Mathematics, Philadelphia (2012)CrossRefzbMATHGoogle Scholar
  5. 5.
    Combettes, P.L., Pesquet, J.C.: A Douglas rachford splitting approach to nonsmooth convex variational signal recovery. IEEE J. Sel. Top. Signal Process. 1(4), 564–574 (2007)CrossRefGoogle Scholar
  6. 6.
    De Bonet, J.S.: Multiresolution sampling procedure for analysis and synthesis of texture images. In: SIGGRAPH, pp. 361–368. ACM (1997)Google Scholar
  7. 7.
    Efros, A.A., Freeman, W.T.: Image quilting for texture synthesis and transfer. In: SIGGRAPH, pp. 341–346. ACM (2001)Google Scholar
  8. 8.
    Efros, A.A., Leung, T.K.: Texture synthesis by non-parametric sampling. In: IEEE International Conference on Computer Vision, pp. 1033–1038 (1999)Google Scholar
  9. 9.
    Ferradans, S., Papadakis, N., Rabin, J., Peyré, G., Aujol, J.-F.: Regularized discrete optimal transport. In: Kuijper, A., Bredies, K., Pock, T., Bischof, H. (eds.) SSVM 2013. LNCS, vol. 7893, pp. 428–439. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-38267-3_36 CrossRefGoogle Scholar
  10. 10.
    Galerne, B., Gousseau, Y., Morel, J.-M.: Random phase textures: theory and synthesis. IEEE Trans. Image Process. 20(1), 257–267 (2011)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Gatys, L., Ecker, A.S., Bethge, M.: Texture synthesis using convolutional neural networks. In: NIPS, pp. 262–270 (2015)Google Scholar
  12. 12.
    G’MIC: Greyc’s magic for image computing (2016).
  13. 13.
    Han, J., Zhou, K., Wei, L.-Y., Gong, M., Bao, H., Zhang, X., Guo, B.: Fast example-based surface texture synthesis via discrete optimization. Vis. Comput. 22(9), 918–925 (2006)CrossRefGoogle Scholar
  14. 14.
    Heeger, D.J., Bergen, J.R.: Pyramid-based texture analysis/synthesis. In: SIGGRAPH, pp. 229–238. ACM (1995)Google Scholar
  15. 15.
    Julesz, B.: Textons, the elements of texture perception, and their interactions. Nature 290(5802), 91–97 (1981)CrossRefGoogle Scholar
  16. 16.
    Julesz, B., Gilbert, E.N., Victor, J.D.: Visual discrimination of textures with identical third-order statistics. Biol. Cybern. 31(3), 137–140 (1978)CrossRefGoogle Scholar
  17. 17.
    Kopf, J., Fu, C.-W., Cohen-Or, D., Deussen, O., Lischinski, D., Wong, T.-T.: Solid texture synthesis from 2d exemplars. In: SIGGRAPH. ACM (2007)Google Scholar
  18. 18.
    Kwatra, V., Schödl, A., Essa, I., Turk, G., Bobick, A.: Graphcut textures: image and video synthesis using graph cuts. In: SIGGRAPH, pp. 277–286. ACM (2003)Google Scholar
  19. 19.
    Kwatra, V., Essa, I., Bobick, A., Kwatra, N.: Texture optimization for example-based synthesis. In: SIGGRAPH, pp. 795–802. ACM (2005)Google Scholar
  20. 20.
    Lefebvre, S., Hoppe, H.: Parallel controllable texture synthesis. In: SIGGRAPH, pp. 777–786. ACM (2005)Google Scholar
  21. 21.
    Liang, L., Liu, C., Xu, Y.-Q., Guo, B., Shum, H.-Y.: Real-time texture synthesis by patch-based sampling. ACM Trans. Graph. 20(3), 127–150 (2001)CrossRefGoogle Scholar
  22. 22.
    Portilla, J., Simoncelli, E.P.: A parametric texture model based on joint statistics of complex wavelet coefficients. Int. J. Comput. Vision 40(1), 49–71 (2000)CrossRefzbMATHGoogle Scholar
  23. 23.
    Raad, L., Galerne, B.: Efros and Freeman image quilting algorithm for texture synthesis. In: IPOL (2016)Google Scholar
  24. 24.
    Rabin, J., Ferradans, S., Papadakis, N.: Adaptive color transfer with relaxed optimal transport. In: IEEE ICIP, pp. 4852–4856 (2014)Google Scholar
  25. 25.
    Rabin, J., Peyré, G.: Wasserstein regularization of imaging problem. In: IEEE ICIP, pp. 1541–1544 (2011)Google Scholar
  26. 26.
    Rabin, J., Peyré, G., Delon, J., Bernot, M.: Wasserstein barycenter and its application to texture mixing. In: Bruckstein, A.M., Haar Romeny, B.M., Bronstein, A.M., Bronstein, M.M. (eds.) SSVM 2011. LNCS, vol. 6667, pp. 435–446. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-24785-9_37 CrossRefGoogle Scholar
  27. 27.
    Tartavel, G., Gousseau, Y., Peyré, G.: Constrained sparse texture synthesis. In: Kuijper, A., Bredies, K., Pock, T., Bischof, H. (eds.) SSVM 2013. LNCS, vol. 7893, pp. 186–197. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-38267-3_16 CrossRefGoogle Scholar
  28. 28.
    Tartavel, G., Gousseau, Y., Peyré, G.: Variational texture synthesis with sparsity and spectrum constraints. JMIV 52(1), 124–144 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Vasconcelos, C.N., Rosenhahn, B.: Bipartite graph matching computation on GPU. In: Cremers, D., Boykov, Y., Blake, A., Schmidt, F.R. (eds.) EMMCVPR 2009. LNCS, vol. 5681, pp. 42–55. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-03641-5_4 CrossRefGoogle Scholar
  30. 30.
    Wei, L.-Y., Lefebvre, S., Kwatra, V., Turk, G.: State of the art in example-based texture synthesis. In: Eurographics - State of the Art Reports (2009)Google Scholar
  31. 31.
    Wei, L.Y., Levoy, M.: Fast texture synthesis using tree-structured vector quantization. In: SIGGRAPH, pp. 479–488. ACM (2000)Google Scholar
  32. 32.
    Xia, G.-S., Ferradans, S., Peyré, G., Aujol, J.-F.: Synthesizing and mixing stationary Gaussian texture models. SIAM J. Imaging Sci. 8(1), 476–508 (2014)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Jorge Gutierrez
    • 1
    Email author
  • Julien Rabin
    • 2
  • Bruno Galerne
    • 3
  • Thomas Hurtut
    • 1
  1. 1.Polytechnique MontréalMontréalCanada
  2. 2.Normandie Univ., ENSICAEN, CNRS, GREYCCaenFrance
  3. 3.Laboratoire MAP5, Université Paris Descartes and CNRS, Sorbonne Paris CitéParisFrance

Personalised recommendations