Convolutional Feature Learning and CNN Based HMM for Arabic Handwriting Recognition

  • Mustapha AmrouchEmail author
  • Mouhcine Rabi
  • Youssef Es-Saady
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10884)


In this paper, we present a model CNN based HMM for Arabic handwriting word recognition. The HMM have proved a powerful to model the dynamics of handwriting. Meanwhile, the CNN have achieved impressive performance in many computer vision tasks, including handwritten characters recognition. In this model, the trainable classifier of CNN is replacing by the HMM classifier. CNN works as a generic feature extractor and HMM performs as a recognizer. The suggested system outperforms a basic HMM based on handcrafted features. Experiments have been conducted on the well-known IFN/ENIT database. The results obtained show the robustness of the proposed approach.


Handwriting recognition Convolutional Neural Networks Hidden Markov Models 


  1. 1.
    Goodfellow, I.J., Bulatov, Y., Ibarz, J., Arnoud, S., Shet, V.: Multi-digit number recognition from street view imagery using deep convolutional neural networks. In: ICLR (2014)Google Scholar
  2. 2.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems 25 (NIPS 2012)Google Scholar
  3. 3.
    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, S., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Li, F.F.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Hamdani, M., Doetsch, P., Kozielski, M., El-Desoky Mousaand, A., Ney, H.: The RWTH large vocabulary arabic handwriting recognition system. In: 11th IAPR International Workshop on Document Analysis Systems, pp. 111–115 (2014)Google Scholar
  5. 5.
    Graves, A.: Supervised Sequence Labeling with Recurrent Neural Networks. Studies in Computational Intelligence. Springer, Heidelberg (2012). Scholar
  6. 6.
    Plotz, T., Fink, G.A.: Markov models for offline handwriting recognition: a survey. Int. J. Doc. Anal. Recogn. 12(4), 269–298 (2009)CrossRefGoogle Scholar
  7. 7.
    Espana-Boquera, S., Castro-Bleda, M.J., Gorbe-Moya, J., Zamora-Martınez, F.: Improving offline handwritten text recognition with Hybrid HMM/ANN models. IEEE Trans. Pattern Anal. Mach. Intell. 33(4), 767–779 (2011)CrossRefGoogle Scholar
  8. 8.
    Elleuch, M., Maalej, R., Kherallah, M.: A new design based-SVM of the CNN classifier architecture with dropout for offline arabic handwritten recognition. Procedia Comput. Sci. 80, 1712–1723 (2016). Online publication date 1 Jan 2016CrossRefGoogle Scholar
  9. 9.
    Ciresan, D.C., Meier, U., Gambardella, L.M., Schmidhuber, J.: Deep big simple neural nets excel on handwritten digit recognition CoRR abs/1003.0358 (2010)Google Scholar
  10. 10.
    Kozielski, M., Doetsch, P., Ney, H.: Improvements in RWTH’s system for off-line handwriting recognition. In: International Conference on Document Analysis and Recognition (ICDAR2013), pp. 935–939 (2013)Google Scholar
  11. 11.
    Xue, H., Govindaraju, V.: Hidden Markov models combining discrete symbols and continuous attributes in handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28(3), 458–462 (2006)CrossRefGoogle Scholar
  12. 12.
    Ahmad, I., Fink, G., Mahmoud, S., et al.: Improvements in sub-character hmm model based arabic text recognition. In: 2014 14th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 537–542. IEEE (2014)Google Scholar
  13. 13.
    Alabodi, J., Li, X.: An effective approach to offline Arabic handwriting recognition. Int. J. Artif. Intell. Appl. 4(6), 1 (2013)Google Scholar
  14. 14.
    Azeem, S.A., Ahmed, H.: Effective technique for the recognition of offline arabic handwritten words using hidden markov models. Int. J. Docum. Anal. Recogn. (IJDAR) 16(4), 399–412 (2013)CrossRefGoogle Scholar
  15. 15.
    El Abed, H., Margner, V.: ICDAR 2009-Arabic handwriting recognition competition. Int. J. Docum. Anal. Recogn. (IJDAR) 14(1), 3–13 (2011)CrossRefGoogle Scholar
  16. 16.
    Pechwitz, M., Maddouri, S.S., Märgner, V., Ellouze, N., Amiri, H.: IFN/ENIT-database of handwritten Arabic words. In: The 7th CIFED 2002, Hammamet, Tunis, 21–23 October 2002 (2002)Google Scholar
  17. 17.
    Lawgali, A., Angelova, M., Bouridane, A.: A framework for arabic handwritten recognition based on segmentation. Int. J. Hybrid Inf. Technol. 7(5), 413–428 (2014)CrossRefGoogle Scholar
  18. 18.
    Parvez, M.T., Mahmoud, S.A.: Offline Arabic handwritten text recognition: a survey. ACM Comput. Surv. 45(2), 23–35 (2013)CrossRefGoogle Scholar
  19. 19.
    Rabi, M., Amrouch, M., Mahani, Z., Mammass, D.: Recognition of cursive Arabic handwritten text using embedded training based on HMMs. In: International Conference on Engineering & MIS (ICEMIS), September 2016, INSPEC Accession Number: 16467172. IEEE.
  20. 20.
    Bluche, T., Ney, H., Kermorvant, C.: Tandem HMM with convolutional neural network for handwritten word recognition. In: 38th International Conference on Acoustics Speech and Signal Processing (ICASSP2013), pp. 2390–2394 (2013)Google Scholar
  21. 21.
    Bluche, T., Ney, H., Kermorvant, C.: Feature extraction with convolutional neural networks for handwritten word recognition. In: 12th International Conference on Document Analysis and Recognition (ICDAR2013) (2013)Google Scholar
  22. 22.
    Le Cun, Y., Kavukcuoglu, K., Farabet, C.: Convolutional networks and applications in vision. In: International Symposium on Circuits and Systems, pp. 253–256, May 2010Google Scholar
  23. 23.
    Mohamad, R.A.-H., Likforman-Sulem, L., Mokbel, C.: Combining slanted-frame classifiers for improved HMM-based Arabic handwriting recognition. In: IEEE PAMI, vol. 31, no. 7, pp. 1165–1177 (2009)Google Scholar
  24. 24.
    Sutskever, I., Martens, J., Dahl, G., Hinton, G.: On the importance of initialization and momentum in deep learning. JMLR W&CP 28(3), 1139–1147 (2013)Google Scholar
  25. 25.
    Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., Le-Cun, Y.: OverFeat: integrated recognition, localization and detection using convolutional network. CoRR (2013)Google Scholar
  26. 26.
    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. CoRR abs/1409.4842 (2014)Google Scholar
  27. 27.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv technical report (2014)Google Scholar
  28. 28.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition (2015).
  29. 29.
    Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML 2010), pp. 807–814 (2010)Google Scholar
  30. 30.
    Le Cun, Y., Bottou, L., Bengio, Y.: Reading checks with multilayer graph transformer networks. In: International Conference on Acoustics, Speech, and Signal Processing (1997)Google Scholar
  31. 31.
    Bengio, Y., Boulanger-Lewandowski, N., Pascanu, R.: Advances in optimizing recurrent networks. CoRR abs/1212.0901 (2012).
  32. 32.
    Baum, L.E., Petrie, T., Soules, G., Weiss, N.: A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Ann. Math. Stat. 41(1), 164–171 (1970). Scholar
  33. 33.
    Forney Jr., G.D.: The Viterbi algorithm. Proc. IEEE 61(3), 268–278 (1973). Scholar
  34. 34.
  35. 35.
    Young, S., et al.: The HTK Book V3.4. Cambridge University Press, Cambridge (2006)Google Scholar
  36. 36.
    Alkhateeb, J.H., Ren, J., Jiang, J., Al-Muhtaseb, H.: Offline handwritten Arabic cursive text recognition using hidden markov models and re-ranking. Pattern Recogn. Lett. 32, 1081–1088 (2011)CrossRefGoogle Scholar
  37. 37.
    Maqqor, A. Halli, A., Satori, K., Tairi, H.: Off-line recognition handwriting combination of mutiple classifiers, In: 3rd International IEEE Colloquium on Information Science and Technology, IEEE CIST 2014, October 2014Google Scholar
  38. 38.
    El Moubtahij, H., Akram, H., Satori, K.: Using features of local densities, statistics and HMM toolkit (HTK) for offline Arabic handwriting text recognition (2016)Google Scholar
  39. 39.
    Jayech, K., Mahjoub, M.A., Amara, N.E.: Arabic handwritten word recognition based on dynamic bayesian network (2016)Google Scholar
  40. 40.
    Giménez, A., Khoury, I., Andrés-Ferrer, J., Juan, A.: Handwriting word recognition using windowed bernoulli HMMs. Pattern Recogn. Lett 35, 149–156 (2012). Article in PressCrossRefGoogle Scholar
  41. 41.
    Hamdani, M., El Abed, H., Kherallah, M., Alimi, A.: Combining multiple HMMs using online and off-line features for off-line Arabic handwriting recognition. In: Proceedings of the 10th International Conference on Document Analysis and Recognition, pp. 201–205 (2009)Google Scholar
  42. 42.
    Kessentini, Y., Paquet, T., Ben Hamadou, A.: Off-line handwritten word recognition using multistream hidden Markov models. Pattern Recogn. Lett. 31, 60–70 (2010)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Mustapha Amrouch
    • 1
    Email author
  • Mouhcine Rabi
    • 2
  • Youssef Es-Saady
    • 2
  1. 1.Laboratory IRF-SIC, Faculty of SciencesIbn Zohr UniversityAgadirMorocco
  2. 2.Laboratory IRF-SICIbn Zohr UniversityAgadirMorocco

Personalised recommendations