A novel CNN structure for fine-grained classification of Chinese calligraphy styles

  • Jiulong ZhangEmail author
  • Mingtao Guo
  • Jianping Fan
Original Paper


Chinese calligraphy is a valuable cultural heritage belonging to the world. It is liked by many people, and our mission is to endeavor to pursue calligraphy and make contributions to the business with technical means. The automatic recognition of the styles of calligraphy by image processing techniques has important meaning in arts collection and auction, etc. Traditional feature operators have some drawbacks that leave room for modern methods like convolutional neural network (CNN). However, most of the studies focus on the classification of five basic fonts that is somewhat different from styles. In this paper, four kinds of styles belonging to standard font are classified with a novel CNN structure where two squeeze-and-excitation modules that emphasize informative feature maps and suppress useless features are embedded after convolution layers, and a Haar transform layer that fuses the features is imposed before softmax layer. Experiment result shows the significance of the proposed structure over other networks in both font and style classification.


Deep learning Convolutional neural network (CNN) Chinese calligraphy Styles classification 



This work is support by National Key Research and Development Plan (No. 2017YFB1402103), Xi’an Science and Technology Bureau Project (201805037YD15CG21(6)). The authors are grateful for the anonymous reviewers for their significant comments.


  1. 1.
    Yao, C., Cheng, G.: Approximative Bayes optimality linear discriminant analysis for Chinese handwriting character recognition. Neurocomputing 207, 346–353 (2016)CrossRefGoogle Scholar
  2. 2.
    Yang, W., Jin, L., Xie, Z., et al.: Improved deep convolutional neural network for online handwritten Chinese character recognition using domain-specific knowledge. In: 2015 13th international conference on document analysis and recognition (ICDAR). IEEE, pp. 551–555 (2015)Google Scholar
  3. 3.
    Gao, P., Wu, J., Yuan, L., et al.: Fast Chinese calligraphic character recognition with large-scale data. Multimed. Tools Appl. 74(17), 7221–7238 (2015)CrossRefGoogle Scholar
  4. 4.
    Gao, P., Wu, J., Yuan, L., et al.: Fast Chinese calligraphic character recognition with large-scale data. Multimed. Tools Appl. 74(17), 7221–7238 (2015)CrossRefGoogle Scholar
  5. 5.
    Tao, D., Lin, X., Jin, L., et al.: Principal component 2-D long short-term memory for font recognition on single Chinese characters. IEEE Trans. Cybern. 46(3), 756 (2016)CrossRefGoogle Scholar
  6. 6.
    Zhang, X., Nagy, G.: Style comparisons in calligraphy. Doc. Recognit. Retr. XIX 8297(2), 263–271 (2012)Google Scholar
  7. 7.
    Zhang, Y., Liu, Y., He, J., et al.: Recognition of calligraphy style based on global feature defontor. In: IEEE International Conference on Multimedia and Expo. IEEE, pp. 1–6 (2013)Google Scholar
  8. 8.
    Gao, P., Wu, J., Lin, Y., et al.: Fast image-based chinese calligraphic character retrieval on large scale data. In: Acm/ieee-Cs Joint Conference on Digital Libraries. IEEE Press, pp. 211–218 (2014)Google Scholar
  9. 9.
    Shi, Z., Xu, B., Zheng, X., et al.: A Chinese character structure preserved denoising method for Chinese tablet calligraphy document images based on KSVD dictionary learning. Multimed. Tools Appl. 76(13), 1–16 (2016)Google Scholar
  10. 10.
    Imdad, A., Bres, S., Eglin, V., et al.: Writer identification using steered hermite features and SVM. In: International Conference on Document Analysis and Recognition. IEEE, pp. 839–843 (2017)Google Scholar
  11. 11.
    Vásquez, J.L., Travieso, C.M., Alonso, J.B.: Using calligraphies features for off line writer identification. In: International Carnahan Conference on Security Technology. IEEE, pp. 1–6 (2014)Google Scholar
  12. 12.
    Sun, R., Lian, Z., Tang, Y., et al.: Aesthetic visual quality evaluation of Chinese handwritings. In: International Conference on Artificial Intelligence. AAAI Press, pp. 2510–2516 (2015)Google Scholar
  13. 13.
    Hou, S., Xu, P.: An algorithm of calligraphy beautification based on improved velocity and width model. In: International Conference on Computer Science. IEEE, pp. 124–127 (2015)Google Scholar
  14. 14.
    Liu, L., Xia, W., Jin, L., et al.: A Kai style contour beautification method for Chinese handwriting characters. In: IEEE International Conference on Systems Man and Cybernetics. IEEE, pp. 3644–3649 (2010)Google Scholar
  15. 15.
    Xiao, J., Xiao, J., Xiao, J.: Automatic generation of large-scale handwriting fonts via style learning. In: SIGGRAPH ASIA 2016 Technical Briefs. ACM, p. 12 (2016)Google Scholar
  16. 16.
    Zhang, S.L., Xie, Z.K., Qi, Y.: Transformation of Chinese calligraphy style based on feature matrix editing. In; International Conference on Computer Science and Network Technology. IEEE, pp. 159–162 (2016)Google Scholar
  17. 17.
    Jie, C., Ma, Y.H.: Novel Chinese calligraphy style generation based on curve analogy. International Workshop on Intelligent Systems and Applications. IEEE, pp. 1–3 (2011)Google Scholar
  18. 18.
    Zhuang, Y., Lu, W., Wu, J.: Latent Style Model: Discovering Writing Styles for Calligraphy Works. Academic Press, Inc., Cambridge (2009)Google Scholar
  19. 19.
    Huang, S., Zhong, Z., Jin, L., et al.: DropRegion training of inception font network for high-performance Chinese font recognition[J]. Pattern Recogn. 77, 395–411 (2018)CrossRefGoogle Scholar
  20. 20.
    Wang, X., Girshick, R., Gupta, A., et al.: Non-local Neural Networks, p. 10 (2017). arXiv preprint arXiv:1711.07971
  21. 21.
    Hu, J., Shen, L., Sun, G.: Squeeze-and-Excitation Networks, p. 7 (2017). arXiv preprint arXiv:1709.01507
  22. 22.
    Szegedy, C., Ioffe, S., Vanhoucke, V. et al: Inception-v4, inception-resnet and the impact of residual connections on learning. In: AAAI, vol. 4, no 12 (2017)Google Scholar
  23. 23.
    Huang, G., Liu, Z., Van Der Maaten, L., et al.: Densely connected convolutional networks[C]. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700–4708 (2017)Google Scholar
  24. 24.
    Xie, S., Girshick, R., Dollár, P., et al.: Aggregated residual transformations for deep neural networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp. 5987–5995 (2017)Google Scholar
  25. 25.
    Haar, A.: Zur theorie der orthogonalen funktionensysteme. Math. Ann. 69(3), 331–371 (1910)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Daugman, J.G.: Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. JOSA A 2(7), 1160–1169 (1985)CrossRefGoogle Scholar
  27. 27.
    Deepu, V., Madhvanath, S., Ramakrishnan, A.G.: Principal component analysis for online handwritten character recognition. In: International Conference on Pattern Recognition, vol. 2, pp. 327–330. IEEE (2004)Google Scholar
  28. 28.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005, vol. 1, pp. 886–893. IEEE (2005)Google Scholar
  29. 29.
    Lécun, Y., Bottou, L., Bengio, Y., et al.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (2001)CrossRefGoogle Scholar
  30. 30.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60(2), 2012 (2013)Google Scholar
  31. 31.
    Simonyan, K., Zisserman, A.,: Very deep convolutional networks for large-scale image recognition[J]. arXiv preprint arXiv:1409.1556 (2014)
  32. 32.
    Szegedy, C., Liu, W., Jia, Y., et al.: Going deeper with convolutions. In: Computer Vision and Pattern Recognition, pp. 1–9. IEEE (2015)Google Scholar
  33. 33.
    He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition[C]. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778 (2016)Google Scholar
  34. 34.
    Deng, J., Dong, W., Socher, R., et al.: ImageNet: a large-scale hierarchical image database. IEEE Conference on IEEE Computer Vision and Pattern Recognition, 2009. CVPR 2009, pp. 248–255 (2009)Google Scholar
  35. 35.
    Xu, K., Ba, J., Kiros, R.,: Show, attend and tell: neural image caption generation with visual attention[C]. In: International conference on machine learning, pp. 2048-2057 (2015)Google Scholar
  36. 36.
    Zhang, H., Goodfellow, I., Metaxas, D., et al.: Self-Attention Generative Adversarial Networks (2018). arXiv preprint arXiv:1805.08318
  37. 37.
    Oyallon, E., Mallat, S.: Deep roto-translation scattering for object classification. In: Computer Vision and Pattern Recognition. IEEE, pp. 2865–2873 (2015)Google Scholar
  38. 38.
    Oyallon, E., Belilovsky, E., Zagoruyko, S.: Scaling the scattering transform: deep hybrid networks[C]. In: Proceedings of the IEEE international conference on computer vision, pp. 5618–5627 (2017)Google Scholar
  39. 39.
    Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10), pp. 807–814 (2010)Google Scholar
  40. 40.
    Ioffe, S., Szegedy, C.: Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift (2015). arXiv preprint arXiv:1502.03167
  41. 41.
    Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323(6088), 533 (1986)CrossRefzbMATHGoogle Scholar
  42. 42.
    Abadi, M., Barham, P., Chen, J., et al.: Tensorflow: a system for large-scale machine learning. In: OSDI, vol. 16, pp. 265–283 (2016)Google Scholar
  43. 43.
    Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)zbMATHGoogle Scholar
  44. 44.
    Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. J. Mach. Learn. Res. 9, 249–256 (2010)Google Scholar
  45. 45.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization[J]. arXiv preprint arXiv:1412.6980 (2014)
  46. 46.
    Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(2605), 2579–2605 (2008)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Institute of Computer Science and EngineeringXi’an University of TechnologyXi’anChina
  2. 2.Shaanxi Key Laboratory for Network Computing and Security TechnologyXi’anChina
  3. 3.University of North Carolina at CharlotteCharlotteUSA

Personalised recommendations