Advertisement

QuiltGAN: An Adversarially Trained, Procedural Algorithm for Texture Generation

  • Renato Barros ArantesEmail author
  • George VogiatzisEmail author
  • Diego FariaEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11754)

Abstract

We investigate a generative method that synthesises high-resolution images based on a single constraint source image. Our approach consists of three types of conditional deep convolutional generative adversarial networks (cDCGAN) that are trained to generate samples of an image patch conditional on the surrounding image regions. The cDCGAN discriminator evaluates the realism of the generated sample concatenated with the surrounding pixels that were conditioned on. This encourages the cDCGAN generator to create image patches that seamlessly blend with their surroundings while maintaining the randomisation of the standard GAN process. After training, the cDCGANs recursively generate a sequence of samples which are then stitched together to synthesise a larger image. Our algorithm is able to produce a nearly infinite collection of variations of a single input image that have enough variability while preserving the essential large-scale constraints. We test our system on several types of images, including urban landscapes, building facades and textures, comparing very favourably against standard image quilting approaches.

Keywords

GAN Procedural generation Inpainting 

References

  1. 1.
    Bergmann, U., Jetchev, N., Vollgraf, R.: Learning texture manifolds with the periodic spatial GAN. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 70, pp. 469–477. PMLR, International Convention Centre, Sydney, Australia, 06–11 August 2017Google Scholar
  2. 2.
    Demir, U., Ünal, G.B.: Patch-based image inpainting with generative adversarial networks. CoRR abs/1803.07422 (2018). http://arxiv.org/abs/1803.07422
  3. 3.
    Efros, A.A., Freeman, W.T.: Image quilting for texture synthesis and transfer. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 2001, pp. 341–346. ACM, New York, NY, USA (2001)Google Scholar
  4. 4.
    Gatys, L.A., Ecker, A.S., Bethge, M.: Texture synthesis using convolutional neural networks. In: Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1, NIPS 2015, pp. 262–270. MIT Press, Cambridge, MA, USA (2015)Google Scholar
  5. 5.
    Goodfellow, I.: NIPS 2016 Tutorial: Generative Adversarial Networks (2016)Google Scholar
  6. 6.
    Goodfellow, I., et al.: Generative adversarial nets. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 27, pp. 2672–2680. Curran Associates, Inc. (2014)Google Scholar
  7. 7.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June 2016, pp. 770–778 (2016)Google Scholar
  8. 8.
    Iizuka, S., Simo-Serra, E., Ishikawa, H.: Globally and locally consistent image completion. ACM Trans. Graph. 36(4), 107:1–107:14 (2017)CrossRefGoogle Scholar
  9. 9.
    Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5967–5976 (2017)Google Scholar
  10. 10.
    Jetchev, N., Bergmann, U., Vollgraf, R.: Texture synthesis with spatial generative adversarial networks. CoRR abs/1611.08207 (2016)Google Scholar
  11. 11.
    Mirza, M., Osindero, S.: Conditional generative adversarial nets. CoRR abs/ 1411.1784 (2014)Google Scholar
  12. 12.
    Pathak, D., Krähenbühl, P., Donahue, J., Darrell, T., Efros, A.: Context encoders: feature learning by inpainting (2016)Google Scholar
  13. 13.
    Portilla, J., Simoncelli, E.P.: A parametric texture model based on joint statistics of complex wavelet coefficients. Int. J. Comput. Vision 40(1), 49–70 (2000)CrossRefGoogle Scholar
  14. 14.
    Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks abs/1511.06434 (2016)Google Scholar
  15. 15.
    Radim Tyleček, R.Š.: Spatial pattern templates for recognition of objects with regular structure. In: Proceedings GCPR, Saarbrucken, Germany (2013)Google Scholar
  16. 16.
    Ratner, A.J., Ehrenberg, H., Hussain, Z., Dunnmon, J., Ré, C.: Learning to compose domain-specific transformations for data augmentation. In: Advances in Neural Information Processing Systems, pp. 3236–3246 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Aston UniversityBirminghamUK

Personalised recommendations