Advertisement

The Journal of Supercomputing

, Volume 75, Issue 4, pp 1922–1940 | Cite as

Multi-scale multi-class conditional generative adversarial network for handwritten character generation

  • Jin Liu
  • Chenkai Gu
  • Jin Wang
  • Geumran Youn
  • Jeong-Uk KimEmail author
Article

Abstract

Handwritten character generation is a popular research topic with various applications. Various methods have been proposed in the literatures which are based on methods such as pattern recognition, machine learning, deep learning or others. However, seldom method could generate realistic and natural handwritten characters with a built-in determination mechanism to enhance the quality of generated image and make the observers unable to tell whether they are written by a person. To address these problems, in this paper, we proposed a novel generative adversarial network, multi-scale multi-class generative adversarial network (MSMC-CGAN). It is a neural network based on conditional generative adversarial network (CGAN), and it is designed for realistic multi-scale character generation. MSMC-CGAN combines the global and partial image information as condition, and the condition can also help us to generate multi-class handwritten characters. Our model is designed with unique neural network structures, image features and training method. To validate the performance of our model, we utilized it in Chinese handwriting generation, and an evaluation method called mean opinion score (MOS) was used. The MOS results show that MSMC-CGAN achieved good performance.

Keywords

GAN CGAN Generate model Handwritten character 

Notes

Acknowledgements

This work was supported by the State Oceanic Administration China research fund Project (201305026), the NSFC (61772454), and by the open research fund of the Key Lab of Broadband Wireless Communication and Sensor Network Technology (Nanjing University of Posts and Telecommunications), Ministry of Education. Prof. Jeong-Uk Kim is the corresponding author.

References

  1. 1.
    Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp 2672–2680Google Scholar
  2. 2.
    Mirza M, Osindero S (2014) Conditional generative adversarial nets. Comput Sci 2672–2680. arXiv:1411.1784
  3. 3.
    Lake BM, Salakhutdinov R, Tenenbaum JB (2015) Human-level concept learning through probabilistic program induction. Science 350(6266):1332–1338MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Haines TSF, Mac Aodha O, Brostow GJ (2016) My text in your handwriting. ACM Trans Gr 35(3):26CrossRefGoogle Scholar
  5. 5.
    Xu S, Jin T, Jiang H, Lau FCM (2009) Automatic generation of personal chinese handwriting by capturing the characteristics of personal handwriting. In: Conference on Innovative Applications of Artificial Intelligence, Pasadena, California, USA. DBLP, 14–16 July 2009Google Scholar
  6. 6.
    Lin JW, Hong CY, Chang R, Wang YC, Lin SY, Ho JM (2015) Complete font generation of Chinese characters in personal handwriting style. In: IEEE Computing and Communications Conference, pp 1–5Google Scholar
  7. 7.
    Kuroiwa T, Shin J (2011) Discovery of efficient chinese characters for handwritten-style font generation. Int J Digit Content Technol Appl 5(12):1–10CrossRefGoogle Scholar
  8. 8.
    Xiao J, Xiao J, Xiao J (2016) Automatic generation of large-scale handwriting fonts via style learning. In: SIGGRAPH ASIA 2016 Technical Briefs, ACM, New York, p 12Google Scholar
  9. 9.
    Myronenko A, Song X (2010) Point set registration: coherent point drift. IEEE Trans Pattern Anal Mach Intell 32(12):2262–2275CrossRefGoogle Scholar
  10. 10.
    Zhang XY, Yin F, Zhang YM, Liu CL, Bengio Y (2017) Drawing and recognizing chinese characters with recurrent neural network. In: IEEE Transactions on Pattern Analysis and Machine IntelligenceGoogle Scholar
  11. 11.
    Ha D, Eck D (2017) A neural representation of sketch drawings. arXiv preprint arXiv:1704.03477
  12. 12.
    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp 1097–1105Google Scholar
  13. 13.
    Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434
  14. 14.
    Zeiler MD, Krishnan D, Taylor GW, Fergus R (2010) Deconvolutional networks. In: IEEE Computer Vision and Pattern Recognition, vol 238, pp 2528–2535Google Scholar
  15. 15.
    Huang R, Zhang S, Li T, He R (2017) Beyond face rotation: global and local perception gan for photorealistic and identity preserving frontal view synthesis. arXiv preprint arXiv:1704.04086
  16. 16.
    Reed S, Akata Z, Yan X, Logeswaran L, Schiele B, Lee H (2016) Generative adversarial text to image synthesis. arXiv preprint arXiv:1605.05396
  17. 17.
    Wang X, Gupta A (2016, October) Generative image modeling using style and structure adversarial networks. In: European Conference on Computer Vision, Springer, Berlin, pp 318–335Google Scholar
  18. 18.
    Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, Shi W (2016) Photo-realistic single image super-resolution using a generative adversarial network. arXiv preprint arXiv:1609.04802
  19. 19.
    Chidambaram M, Qi Y (2017) Style transfer generative adversarial networks: learning to play chess differently. arXiv preprint arXiv:1702.06762
  20. 20.
    Jetchev N, Bergmann U, Vollgraf R (2016) Texture synthesis with spatial generative adversarial networks. arXiv preprint arXiv:1611.08207
  21. 21.
    Li C, Xu K, Zhu J, Zhang B (2017) Triple generative adversarial nets. arXiv preprint arXiv:1703.02291
  22. 22.
    Arjovsky M, Chintala S, Bottou L (2017) Wasserstein gan. arXiv preprint arXiv:1701.07875
  23. 23.
    Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. arXiv preprint arXiv:1703.10593
  24. 24.
    Antipov G, Baccouche M, Dugelay JL (2017) Face aging with conditional generative adversarial networks. arXiv preprint arXiv:1702.01983
  25. 25.
    Zhang H, Sindagi, V, Patel VM (2017) Image de-raining using a conditional generative adversarial network. arXiv preprint arXiv:1701.05957
  26. 26.
    Isola P, Zhu JY, Zhou T, Efros AA (2016) Image-to-image translation with conditional adversarial networks. arXiv preprint arXiv:1611.07004
  27. 27.
    Denton E, Gross S, Fergus R (2016) Semi-supervised learning with context-conditional generative adversarial networks. arXiv preprint arXiv:1611.06430
  28. 28.
    Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp 448–456Google Scholar
  29. 29.
    Zeiler MD, Krishnan D, Taylor GW, Fergus R (2010) Deconvolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010, pp 2528–2535Google Scholar
  30. 30.
    Kingma D, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980
  31. 31.
    Nair V, Hinton GE (2010) Rectified linear units improve restricted Boltzmann machines. In: International Conference on International Conference on Machine Learning, Omnipress, Madison, pp 807–814Google Scholar
  32. 32.
    Li Q, Xu Q, Xiao J, Liu Q, Zhang J (2017) A structure and style model for chinese character dynamic generation. Beijing Daxue Xuebao (Ziran Kexue Ban)/Acta Scientiarum Naturalium Universitatis Pekinensis, vol 53, pp 219–229.  https://doi.org/10.13209/j.0479-8023.2017.048
  33. 33.
    Wang Y, Che W, Xu B (2017) Encoder–decoder recurrent network model for interactive character animation generation. Vis Comput 33(6–8):971–980CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2017

Authors and Affiliations

  • Jin Liu
    • 1
  • Chenkai Gu
    • 1
  • Jin Wang
    • 2
    • 3
  • Geumran Youn
    • 4
  • Jeong-Uk Kim
    • 4
    Email author
  1. 1.College of Information EngineeringShanghai Maritime UniversityShanghaiChina
  2. 2.Key Lab of Broadband Wireless Communication and Sensor Network Technology (Nanjing University of Posts and Telecommunications)Ministry of EducationNanjingChina
  3. 3.College of Information EngineeringYangzhou UniversityYangzhouChina
  4. 4.Department of Electrical EngineeringSangmyung UniversitySeoulKorea

Personalised recommendations