Arbitrary Style Transfer of Facial Image Based on Feed-Forward Network and Its Application in Aesthetic QR Code

  • Shanqing Zhang
  • Shengqi Su
  • Li LiEmail author
  • Jianfeng Lu
  • Ching-Chun Chang
  • Qili Zhou
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 895)


QR code has become essential in daily-life because of the popularity of mobile devices. The visual effect of a conventional QR code is not ideal. Consequently, many good aesthetic algorithms have been proposed. However, both the decoding rate and visual effect of a QR code cannot be guaranteed simultaneously when facial image serves as the background. We propose an arbitrary style transfer of facial image based on feed-forward network as a preprocessing algorithm for an aesthetic QR code. The deep characteristics of content image and style image are unified in the same layer of convolutional neural networks in our style transfer network. Styles are changed. The result of style transfer is restricted with semantic segmentation result, color uniform regularization of facial image and repeating restriction similarity constraints. Experimental results show that both the decoding rate and visual effect of a QR code are guaranteed when our method is used in background preprocessing.


Arbitrary style transfer Feed-forward network Facial image Aesthetic QR code 



This work was mainly supported by National Natural Science Foundation of China (No. 61370218, No. GG19F020033), Public Welfare Technology and Industry Project of Zhejiang Provincial Science Technology Department (No. 2016C31081, No. LGG18F020013, No. LGG19F020016).


  1. 1.
    Xu, M., et al.: ART-UP: a novel method for generating scanning-robust aesthetic QR codes (2018)Google Scholar
  2. 2.
    Chen, T.Q., Schmidt, M.: Fast patch-based style transfer of arbitrary style (2016)Google Scholar
  3. 3.
    Ono, S., Morinaga, K., Nakayama, S.: Two-dimensional barcode decoration based on real-coded genetic algorithm. In: Evolutionary Computation, pp. 1068–1073. IEEE (2008)Google Scholar
  4. 4.
    Samretwit, D., Wakahara, T.: Measurement of reading characteristics of multiplexed image in QR code. In: International Conference on Intelligent Networking & Collaborative Systems, pp. 552–557. IEEE Computer Society (2011)Google Scholar
  5. 5.
    Li, L., Qiu, J., Lu, J., Chang, C.C.: An aesthetic QR code solution based on error correction mechanism. J. Syst. Softw. S016412121500148X (2016)Google Scholar
  6. 6.
    Fujita, K., Kuribayashi, M., Morii, M.: Expansion of image displayable area in design QR code and its applications. In: Proceedings of the Forum on Information Technology Papers, vol. 10, no. 4, pp. 517–520 (2011)Google Scholar
  7. 7.
    Lin, S.S., Hu, M.C., Lee, C.H., Lee, T.Y.: Efficient QR code beautification with high quality visual content. IEEE Trans. Multimed. 17(9), 1515–1524 (2015)CrossRefGoogle Scholar
  8. 8.
    Cox, R.: QArt Codes. Accessed 20 Jan 2017
  9. 9.
    Li, L., Li, Y., Wang, B., Lu, J., Zhang, S., Yuan, W., Wang, S., Chang, C.C.: A new aesthetic QR code algorithm based on salient region detection and SPBVM (2017)Google Scholar
  10. 10.
    Bonet, J.S.D.: Multiresolution sampling procedure for analysis and synthesis of texture images. In: Conference on Computer Graphics and Interactive Techniques, pp. 361–368. ACM Press/Addison-Wesley Publishing Co. (2017)Google Scholar
  11. 11.
    Heeger, D.J., Bergen, J.R.: Pyramid-based texture analysis/synthesis. In: International Conference on Image Processing. Proceedings, pp. 229–238 (1995)Google Scholar
  12. 12.
    Portilla, J., Simoncelli, E.P.: A parametric texture model based on joint statistics of complex wavelet coefficients. Int. J. Comput. Vision 40(1), 49–70 (2000)CrossRefGoogle Scholar
  13. 13.
    Ashikhmin, N.: Fast texture transfer. Comput. Graph. Appl. IEEE 23(4), 38–43 (2003)CrossRefGoogle Scholar
  14. 14.
    Efros, A,A., Freeman, W.T.: Image quilting for texture synthesis and transfer. In: Conference on Computer Graphics and Interactive Techniques, pp. 341–346. ACM (2001)Google Scholar
  15. 15.
    Kwatra, V., Essa, I., Turk, G., et al.: Graphcut textures: image and video synthesis using graph cuts. In: ACM SIGGRAPH, pp. 277–286. ACM (2003)Google Scholar
  16. 16.
    Li, C., Wand, M.: Combining Markov random fields and convolutional neural networks for image synthesis, pp. 2479–2486 (2016)Google Scholar
  17. 17.
    Gatys, L.A., Ecker, A.S., Bethge, M.: A neural algorithm of artistic style. Computer Science (2015)Google Scholar
  18. 18.
    Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation (2018)Google Scholar
  19. 19.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  20. 20.
    Gu, S., Chen, C., Liao, J., Yuan, L.: Arbitrary style transfer with deep feature reshuffle (2018)Google Scholar
  21. 21.
    Lu, J., Lu, J., Lu, J.: StyLit: illumination-guided example-based stylization of 3D renderings. Acm Trans. Graph. 35(4), 92 (2016). Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: International Conference on Neural Information Processing Systems, pp. 1097–1105. Curran Associates Inc. (2012)Google Scholar
  22. 22.
    Luan, F., Paris, S., Shechtman, E., Bala, K.: Deep photo style transfer, pp. 6997–7005 (2017)Google Scholar
  23. 23.
    Johnson, J., Alahi, A., Li, F.F.: Perceptual losses for real-time style transfer and super-resolution, pp. 694–711 (2016)Google Scholar
  24. 24.
    Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., et al.: Microsoft COCO: common objects in context, vol. 8693, pp. 740–755 (2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Shanqing Zhang
    • 1
  • Shengqi Su
    • 1
  • Li Li
    • 1
    Email author
  • Jianfeng Lu
    • 1
  • Ching-Chun Chang
    • 2
  • Qili Zhou
    • 1
  1. 1.School of Computer Science and TechnologyHangzhou Dianzi UniversityHangzhouChina
  2. 2.Department of Computer ScienceUniversity of WarwickCoventryUK

Personalised recommendations