Towards Unsupervised Learning for Arabic Handwritten Recognition Using Deep Architectures

  • Mohamed ElleuchEmail author
  • Najiba Tagougui
  • Monji Kherallah
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9489)


In the pattern recognition field and especially in the Handwriting recognition one, the Deep learning is becoming the new trend in Artificial Intelligence with the sheer size of raw data available nowadays. In this paper, we highlights how Deep Learning techniques can be effectively applied for recognizing Arabic handwritten script, our field of interest, and this by investigating two deep architectures: Deep Belief Network (DBN) and Convolutional Neural Networks (CNN). The two proposed architectures take the raw data as input and proceed with a greedy layer-wise unsupervised learning algorithm. The experimental study has proved promising results which are comparable or even superior to the standard classifiers with an efficiency of DBN over CNN architecture.


Recognition Arabic handwritten script DBN CNN Unsupervised learning 


  1. 1.
    Mohamed, A., Dahl, G., Hinton, G.: Acoustic modeling using deep belief networks. IEEE Trans. Audio Speech Lang. Process. 20(1), 14–22 (2011)CrossRefGoogle Scholar
  2. 2.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  3. 3.
    Zhong, S., Liu, Y., Liu, Y.: Bilinear deep learning for image classification. In: Proceedings of the 19th ACM International Conference on Multimedia, pp. 343–352 (2011)Google Scholar
  4. 4.
    Karpathy, A., Joulin, A., Fei-Fei, L.: Deep fragment embeddings for bidirectional image sentence mapping. In: Computer Vision and Pattern Recognition, pp. 1889–1897 (2014)Google Scholar
  5. 5.
    Ranzato, M., Huang, F., Boureau, Y., LeCun, Y.: Unsupervised learning of invariant feature hierarchies with applications to object recognition. In: Proceedings of Computer Vision and Pattern Recognition Conference (CVPR), pp. 1–8. IEEE Press (2007)Google Scholar
  6. 6.
    Ciresan, D.C., Meier, U., Schmidhuber, J.: Transfer learning for latin and Chinese characters with deep neural networks. In: Proceedings of International Joint Conference on Neural Networks (IJCNN), pp. 1–6. IEEE Press (2012)Google Scholar
  7. 7.
    Ciresan, D.C., Schmidhuber, J.: Multi-Column Deep Neural Networks for Offline Handwritten Chinese Character Classification. IDSIA Technical Report No. IDSIA-05-13 (2013)Google Scholar
  8. 8.
    Salakhutdinov, R., Tenenbaum, J.B., Torralba, A.: Learning with hierarchical-deep models. Pattern Anal. Mach. Intell. 35(8), 1958–1971 (2013)CrossRefGoogle Scholar
  9. 9.
    Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H.: Greedy layer-wise training of deep networks. In: Advances in Neural Information Processing Systems (NIPS), pp. 153–160 (2007)Google Scholar
  10. 10.
    Tagougui, N., Kherallah, M., Alimi, A.M.: Online Arabic handwriting recognition: a survey. Int. J. Doc. Anal. Recogn. (IJDAR) 16(3), 209–226 (2013)CrossRefGoogle Scholar
  11. 11.
    El Abed, H., Margner, V.: Comparison of different preprocessing and feature extraction methods for offline recognition of handwritten Arabic words. In: Ninth International Conference on Document Analysis and Recognition (ICDAR), pp. 974–978 (2007)Google Scholar
  12. 12.
    Al-Hajj Mohamad, R., Likforman-Sulem, L., Mokbel, C.: Combining slanted-frame classifiers for improved HMM-based Arabic handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31, 1165–1177 (2009)CrossRefGoogle Scholar
  13. 13.
    Benouareth, A., Ennaji, A., Sellami, M.: HMMs with explicit state duration applied to handwritten Arabic word recognition. In: 18th International Conference on Pattern Recognition (ICPR), pp. 897–900 (2006)Google Scholar
  14. 14.
    Benouareth, A., Ennaji, A., Sellami, M.: Semi-continuous HMMs with explicit state duration for unconstrained Arabic word modeling and recognition. Pattern Recogn. Lett. 29, 1742–1752 (2008)CrossRefzbMATHGoogle Scholar
  15. 15.
    Chen, J., Cao, H., Prasad, R., Bhardwaj, A., Natarajan, P.: Gabor features for offline arabic handwriting recognition. In: Proceedings of the 9th IAPR International Workshop on Document Analysis Systems (DAS), pp. 53–58 (2010)Google Scholar
  16. 16.
    Hamdi, R., Bouchareb, F., Bedda, M.: Handwritten Arabic character recognition based on SVM Classifier. In: 3rd International Conference on Information and Communication Technologies: From Theory to Applications (ICTTA), pp. 1–4. IEEE Press (2008)Google Scholar
  17. 17.
    Chherawala, Y., Roy, P.P., Cheriet, M.: Feature design for offline Arabic handwriting recognition: handcrafted vs automated?. In: 12th International Conference on Document Analysis and Recognition (ICDAR), pp. 290–294 (2013)Google Scholar
  18. 18.
    Porwal, U., Zhou, Y., Govindaraju, V.: Handwritten Arabic text recognition using deep belief networks. In: 21st International Conference on Pattern Recognition (ICPR), pp. 302–305. IEEE Press (2012)Google Scholar
  19. 19.
    Hinton, G., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Tieleman, T., Hinton, G.: Using fast weights to improve persistent contrastive divergence. In: Proceedings of the 26th Annual International Conference on Machine Learning (ICML), pp. 1033–1040 (2009)Google Scholar
  21. 21.
    Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors nature. Nature 323(9), 533–536 (1986)CrossRefGoogle Scholar
  22. 22.
    Mohamed, A., Sainath, T.N., Dahl, G., Ramabhadran, B., Hinton, G.E., Picheny, M.A.: Deep belief networks using discriminative features for phone recognition. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 5060–5063 (2011)Google Scholar
  23. 23.
    Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Comput. 14(8), 1771–1800 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313, 504–507 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  26. 26.
    LeCun, Y., Huang, F.J., Bottou, L.: Learning methods for generic object recognition with invariance to pose and lighting. In: Proceedings of Computer Vision and Pattern Recognition Conference (CVPR). IEEE Press (2004)Google Scholar
  27. 27.
    Lawgali, A., Angelova, M., Bouridane, A.: HACDB: handwritten Arabic characters database for automatic character recognition. In: European Workshop on Visual Information Processing (EUVIP), pp. 255–259 (2013)Google Scholar
  28. 28.
    Simard, P., Steinkraus, D., Platt, J.C.: Best practices for convolutional neural networks applied to visual document analysis. In: International Conference on Document Analysis and Recognition (ICDAR), pp. 958–962 (2003)Google Scholar
  29. 29.
    Carreira-Perpinan, M.A., Hinton, G.E.: On contrastive divergence learning. In: Proceedings of the tenth International Workshop on Artificial Intelligence and Statistics, pp. 33–40 (2005)Google Scholar
  30. 30.
    Sazal, M.M.R., Biswas, S.K., Amin, M.F., Murase, K.: Bangla handwritten character recognition using deep belief network. In: Proceedings of the 2013 International Conference on Electrical Communication and Information Technology (ECIT), pp. 1–5 (2013)Google Scholar
  31. 31.
    Ciresan, D.C., Meier, U., Gambardella, L.M., Schmidhuber, J.: Convolutional neural network committees for handwritten character classification. In: 11th International Conference on Document Analysis and Recognition (ICDAR), pp. 1135–1139. IEEE Press (2011)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Mohamed Elleuch
    • 1
    • 2
    Email author
  • Najiba Tagougui
    • 3
  • Monji Kherallah
    • 2
  1. 1.National School of Computer Science (ENSI)University of ManoubaManoubaTunisia
  2. 2.Advanced Technologies for Medicine and Signals (ATMS)University of SfaxSfaxTunisia
  3. 3.The Higher Institute of Management of GabesUniversity of GabesGabèsTunisia

Personalised recommendations