One-Shot Learning-Based Handwritten Word Recognition

  • Asish Chakrapani GvEmail author
  • Sukalpa Chanda
  • Umapada Pal
  • David Doermann
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12047)


One-Shot and Few-shot Learning algorithms have emerged as techniques that can imitate a humans ability to learn from very few examples. This is an advantage over traditional deep networks which require a lot of training samples and lack of robustness due to their excessive domain specific discriminators. In this paper, we explore a one-shot learning approach to recognizing handwritten words using Siamese networks to classify the handwritten images at the word level. The Siamese network’s ability to compute similarities between two images is learned using a supervised metric but the fully trained Siamese network can be used to classify new data that has previously not been used to train the network. The model learns to discriminate inputs from a small labelled support set. By using a convolutional architecture we were able to achieve robust results. We also expect that training the system over a larger distributions of data will result in improved general handwritten word classification. Accuracy as high as 92.4% was obtained while performing 5-way one-shot word recognition on a publicly available dataset which is quite high in comparison to the state-of-the-art methods.


One-shot learning Handwriting recognition Siamese Networks Image classification 


  1. 1.
    Altae-Tran, H., Ramsundar, B., Pappu, A.S., Pande, V.: Low data drug discovery with one-shot learning. ACS Central Sci. 3(4), 283–293 (2017)CrossRefGoogle Scholar
  2. 2.
    Barakat, B., Alaasam, R., El-Sana, J.: Word spotting using convolutional siamese network, pp. 229–234 (2018).
  3. 3.
    Biederman, I.: Recognition-by-components: a theory of human understanding. Psychol. Rev. 31(2), 115–147 (1987)CrossRefGoogle Scholar
  4. 4.
    Lake, B., Salakhutdinov, R., Gross, J., Tenenbaum, J.: One shot learning of simple visual concepts. In: Proceedings of the 33rd Annual Conference of the Cognitive Science Society, Boston, USA, vol. 172 (2009)Google Scholar
  5. 5.
    Bromley, J., Guyon, I., LeCun, Y., et al.: Signature verification using a “siamese” time delay neural network. Int. J. Pattern Recogn. Artif. Intell. 7(04), 669–688 (1993)CrossRefGoogle Scholar
  6. 6.
    Chanda, S., Baas, J., Haitink, D., Hamel, S., Stutzmann, D., Schomaker, L.: Zero-shot learning based approach for medieval word recognition using deep-learned features. In: 16th International Conference on Frontiers in Handwriting Recognition, ICFHR, Niagara Falls, NY, USA, pp. 345–350 (2018)Google Scholar
  7. 7.
    Chopra, S., Hadsell, R., LeCun, Y.: Learning a similarity metric discriminatively, with application to face verification. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), 20–26 June 2005, San Diego, CA, USA, pp. 539–546 (2005)Google Scholar
  8. 8.
    Deng, L., Hinton, G., Kingsbury, B.: New types of deep neural network learning for speech recognition and related applications: an overview. In: Proceedings of International Conference on Acoustic, Speech, and Signal processing, Vancouver, Canada, pp. 8599–8603, May 2013Google Scholar
  9. 9.
    Duan, Y., et al.: One-Shot Imitation Learning. In: NIPS, Long Beach, USA, December 2017Google Scholar
  10. 10.
    Fei Fei, L., Fergus, R., Perona, P.: One-shot learning of object categories. IEEE Trans. Pattern Anal. Mach. Intell. 28(4), 594–611 (2006)CrossRefGoogle Scholar
  11. 11.
    Fei-Fei, L.: Knowledge transfer in learning to recognize visual object classes. In: International Conference on Development and Learning (ICDL), Bloomington, Indiana, USA (2006)Google Scholar
  12. 12.
    Fei-Fei, L., Fergus, R., Perona, P.: A Bayesian approach to unsupervised one-shot learning of object categories. In: Proceedings of 9th IEEE International Conference on Computer Vision, Nice, France, pp. 1134–1141, October 2003Google Scholar
  13. 13.
    Fischer, A., Keller, A., Frinken, V., Bunke, H.: Lexicon-free handwritten word spotting using character HMMs. Pattern Recogn. Lett. - PRL 33, 934–942 (2012)CrossRefGoogle Scholar
  14. 14.
    Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2010, 13–15 May 2010, Chia Laguna Resort, Sardinia, Italy, pp. 249–256 (2010)Google Scholar
  15. 15.
    Howe, N.R.: Part-structured inkball models for one-shot handwritten word spotting. In: 2013 12th International Conference on Document Analysis and Recognition, pp. 582–586 (2013)Google Scholar
  16. 16.
    Keskar, N.S., Mudigere, D., Nocedal, J., Smelyanskiy, M., Tang, P.T.P.: On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima. arXiv preprint arXiv:1609.04836 (2016)
  17. 17.
    Koch, G., Zemel, R., Salakhudtdinov, R.: Siamese neural networks for one-shot image recognition. In: Proceedings of the 32nd International Conference on Machine Learning, Lille, France, vol. 37, July 2015Google Scholar
  18. 18.
    Maas, A., Kemp, C.: One-shot learning with Bayesian networks. In: Proceedings of Cognitive Science Society, Amsterdam, Netherlands, August 2009Google Scholar
  19. 19.
    Russakovsky, O., et al.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. (IJCV) 115, 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., Lillicrap, T.: Meta-learning with memory-augmented neural networks. In: Proceedings of the International Conference on Machine Learning (ICML), New York City, USA, pp. 1842–1850, June 2016Google Scholar
  21. 21.
    Silver, D.: Mastering the game of Go with deep neural networks and tree search. Nature 529, 484–489 (2016)CrossRefGoogle Scholar
  22. 22.
    Thorpe, S., Fize, D., Marlot, C.: Speed of processing in the human visual system. Nature 381, 520–522 (1996)CrossRefGoogle Scholar
  23. 23.
    Vinyals, O., Blundell, C., Lillicrap, T.P., Kavukcuoglu, K., Wierstra, D.: Matching networks for one shot learning. CoRR abs/1606.04080 (2016).
  24. 24.
    Wilkinson, T., Lindström, J., Brun, A.: Neural Ctrl-F: segmentation-free query-by-string word spotting in handwritten manuscript collections. In: IEEE International Conference on Computer Vision, ICCV, Venice, Italy, pp. 4443–4452 (2017)Google Scholar
  25. 25.
    Woodward, M., Finn, C.: Active one-shot learning. In: NIPS, Deep Reinforcement Learning Workshop, Barcelona, Spain, December 2016Google Scholar
  26. 26.
    Wu, Y., et al.: Googles Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. arXiv preprint arXiv:1609.08144 (2016)

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Asish Chakrapani Gv
    • 1
    Email author
  • Sukalpa Chanda
    • 2
  • Umapada Pal
    • 3
  • David Doermann
    • 4
  1. 1.Electronics and Communication DepartmentManipal University JaipurJaipurIndia
  2. 2.Centre for Image Analysis, Department of Information TechnologyUppsala UniversityUppsalaSweden
  3. 3.Computer Vision and Pattern Recognition UnitIndian Statistical InstituteKolkataIndia
  4. 4.University at Buffalo, SUNYBuffaloUSA

Personalised recommendations