Bidirectional Language Model for Handwriting Recognition

  • Volkmar Frinken
  • Alicia Fornés
  • Josep Lladós
  • Jean-Marc Ogier
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7626)


In order to improve the results of automatically recognized handwritten text, information about the language is commonly included in the recognition process. A common approach is to represent a text line as a sequence. It is processed in one direction and the language information via n-grams is directly included in the decoding. This approach, however, only uses context on one side to estimate a word’s probability. Therefore, we propose a bidirectional recognition in this paper, using distinct forward and a backward language models. By combining decoding hypotheses from both directions, we achieve a significant increase in recognition accuracy for the off-line writer independent handwriting recognition task. Both language models are of the same type and can be estimated on the same corpus. Hence, the increase in recognition accuracy comes without any additional need for training data or language modeling complexity.


handwriting recognition language models neural networks 


  1. 1.
    Bauer, L.: Manual of Information to Accompany The Wellington Corpus of Written New Zealand English. Technical report, Department of Linguistics, Victoria University, Wellington, New Zealand (1993)Google Scholar
  2. 2.
    Bunke, H., Bengio, S., Vinciarelli, A.: Offline Recognition of Unconstrained Handwritten Texts using HMMs and Statistical Language Models. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(6), 709–720 (2004)CrossRefGoogle Scholar
  3. 3.
    Espana-Boquera, S., Castro-Bleda, M.J., Gorbe-Moya, J., Zamora-Martínez, F.: Improving Offline Handwritten Text Recognition with Hybrid HMM/ANN Models. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(4), 767–779 (2011)CrossRefGoogle Scholar
  4. 4.
    Fiscus, J.: A Post-processing System to Yield Reduced Word Error Rates: Recognizer Output Voting Error Reduction (ROVER). In: Workshop on Automatic Speech Recognition and Understanding, pp. 347–354. IEEE (December 1997)Google Scholar
  5. 5.
    Goodman, J.T.: A Bit of Progress in Language Modeling - Extended Version. Technical Report MSR-TR-2001-72, Microsoft Research, One Microsoft Way Redmond, WA 98052, 8 (2001)Google Scholar
  6. 6.
    Graves, A., Liwicki, M., Fernández, S., Bertolami, R., Bunke, H., Schmidhuber, J.: A novel Connectionist System for Unconstrained Handwriting Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(5), 855–868 (2009)CrossRefGoogle Scholar
  7. 7.
    Jelinek, F.: Stochastic Analysis of Structured Language Modeling. In: Mathematical Foundations of Speech and Language Processing, vol. 138, pp. 37–71. Springer,Google Scholar
  8. 8.
    Johansson, S., Atwell, E., Garside, R., Leech, G.: The tagged lob corpus: Users’ manual. Technical report, The Norwegian Computing Centre for the Humanities (1986)Google Scholar
  9. 9.
    Kucera, H., Francis, W.N.: Manual of Information to accompany A Standard Corpus of Present-Day Edited American English, for use with Digital Computers. Brown University, Department of Linguistics, Providence, Rhode Island, 1964. Revised 1971. Revised and amplified (1979)Google Scholar
  10. 10.
    Marti, U.-V., Bunke, H.: Using a Statistical Language Model to Improve the Performance of an HMM-Based Cursive Handwriting Recognition System. Int. Journal of Pattern Recognition and Artificial Intelligence 15, 65–90 (2001)CrossRefGoogle Scholar
  11. 11.
    Marti, U.V., Bunke, H.: The iam-database: An English Sentence Database for Offline Handwriting Recognition. Int’l Journal on Document Analysis and Recognition 5(1), 39–46 (2002)zbMATHCrossRefGoogle Scholar
  12. 12.
    Plamondon, R., Srihari, S.N.: Online and Off-Line Handwriting Recognition: A Comprehensive Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(1), 63–84 (2000)CrossRefGoogle Scholar
  13. 13.
    Plötz, T., Fink, G.A.: Markov Models for Offline Handwriting Recognition: A Survey. Int’l Journal on Document Analysis and Recognition 12(4), 269–298 (2009)CrossRefGoogle Scholar
  14. 14.
    Rosenfeld, R., Chen, S.F., Zh, X.: Whole-Sentence Exponential Language Models: A Vehicle for Linguistic-Statistical Integration. Computers, Speech and Language 15, 55–73 (2001)CrossRefGoogle Scholar
  15. 15.
    Stolcke, A.: SRILM: An Extensible Language Modeling Toolkit, pp. 901–904 (2002)Google Scholar
  16. 16.
    Stolke, A., König, Y., Weintraub, M.: Explicit Word Error Minimization in N-Best List Rescoring. In: EUROSPEECH, pp. 163–166 (1997)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Volkmar Frinken
    • 1
  • Alicia Fornés
    • 1
  • Josep Lladós
    • 1
  • Jean-Marc Ogier
    • 2
  1. 1.Computer Vision Center, Dept. of Computer ScienceEdifici O, UABSpain
  2. 2.L3i LaboratoryUniversité de La RochelleLa Rochelle Cédex 1France

Personalised recommendations