Skip to main content
SpringerLink
Log in

Handwriting Recognition

  • Chapter
Markov Models for Pattern Recognition

Part of the book series: Advances in Computer Vision and Pattern Recognition ((ACVPR))

Abstract

In contrast to the field of automatic speech recognition where Markov model-based methods currently represent the state-of-the-art, HMMs and n-gram models are still a rather new approach for the recognition of machine-printed or handwritten texts. In this chapter we will present state-of-the-art systems for offline handwriting recognition. In addition to explanations of how Markov model technology is applied, these presentations will also include brief descriptions of the specialized methods for preprocessing and feature extraction used.

The first system presented in this chapter is BBN’s offline HWR system. It can be considered as a typical example for an HMM-based system for the recognition of machine-printed or handwritten script. Afterwards, we will present the offline HWR system of RWTH Aachen University, Aachen, Germany. The chapter concludes with a presentation of our own systems for offline handwriting recognition which include a system based on so-called Bag-of-Features HMMs. This recently proposed extension of the HMM framework works especially well on the challenging task of query-by-example word spotting.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 49.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 89.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Abbreviation for dots per inch.

  2. 2.

    According to [159] no size or height normalization of the line images is performed.

  3. 3.

    This size normalization operation is script dependent. For Roman script the implicit assumption that average character width is correlated with the average distances between contour minima is justified quite well. In contrast, for Arabic script this size normalization technique does not produce useful results.

  4. 4.

    In [317] lexicon free experiments are reported for bi-gram up to 5-gram models.

References

  1. Basha Shaik, M.A., Rybach, D., Hahn, S., Schlüter, R., Ney, H.: Hierarchical hybrid language models for open vocabulary continuous speech recognition using wfst. In: Proc. Workshop on Statistical and Perceptual Audition, Portland, OR, USA, pp. 46–51 (2012)

    Google Scholar 

  2. Bazzi, I., Schwartz, R., Makhoul, J.: An omnifont open-vocabulary OCR system for English and Arabic. IEEE Trans. on Pattern Analysis and Machine Intelligence 21(6), 495–504 (1999)

    Article  Google Scholar 

  3. Caesar, T., Gloger, J.M., Mandler, E.: Preprocessing and feature extraction for a handwriting recognition system. In: Proc. Int. Conf. on Document Analysis and Recognition, Tsukuba Science City, Japan, pp. 408–411 (1993)

    Google Scholar 

  4. Colthurst, T., Kimball, O., Richardson, F., Shu, H., Wooters, C., Iyer, R., Gish, H.: The 2000 BBN Byblos LVCSR system. In: 2000 Speech Transcription Workshop, Maryland (2000)

    Google Scholar 

  5. Dolfing, J.G.A., Haeb-Umbach, R.: Signal representations for Hidden Markov Model based on-line handwriting recognition. In: Proc. Int. Conf. on Acoustics, Speech, and Signal Processing, München, vol. IV, pp. 3385–3388 (1997)

    Google Scholar 

  6. Dreuw, P., Jonas, S., Ney, H.: White-space models for offline Arabic handwriting recognition. In: Proc. Int. Conf. on Pattern Recognition, Tampa, FL, USA, pp. 1–4 (2008)

    Google Scholar 

  7. Dreuw, P., Rybach, D., Heigold, G., Ney, H.: Rwth ocr: a large vocabulary optical character recognition system for Arabic scripts. In: Märgner, V., El Abed, H. (eds.) Guide to OCR for Arabic Scripts, pp. 215–254. Springer, London, UK (2012). Chap. Part II: Recognition

    Chapter  Google Scholar 

  8. Fei-Fei, L., Perona, P.: A Bayesian hierarchical model for learning natural scene categories. In: Proc. IEEE Comp. Soc. Conf. on Computer Vision and Pattern Recognition, vol. 2, pp. 524–531 (2005)

    Google Scholar 

  9. Fink, G.A., Plötz, T.: On appearance-based feature extraction methods for writer-independent handwritten text recognition. In: Proc. Int. Conf. on Document Analysis and Recognition, Seoul, Korea, vol. 2, pp. 1070–1074 (2005)

    Google Scholar 

  10. Fink, G.A., Plötz, T.: Unsupervised estimation of writing style models for improved unconstrained off-line handwriting recognition. In: Proc. Int. Workshop on Frontiers in Handwriting Recognition, La Baule, France, pp. 429–434 (2006)

    Google Scholar 

  11. Fink, G.A., Plötz, T.: On the use of context-dependent modelling units for HMM-based offline handwriting recognition. In: Proc. Int. Conf. on Document Analysis and Recognition, Curitiba, Brazil, vol. 2, pp. 729–733 (2007)

    Google Scholar 

  12. Fink, G.A., Vajda, S., Bhattacharya, U., Parui, S.K., Chaudhuri, B.B.: Online Bangla word recognition using sub-stroke level features and hidden Markov models. In: Proc. Int. Conf. on Frontiers in Handwriting Recognition, Kolkata, India, pp. 393–398 (2010)

    Google Scholar 

  13. Fink, G.A., Wienecke, M.: Experiments in video-based whiteboard reading. In: First Int. Workshop on Camera-Based Document Analysis and Recognition, Seoul, Korea, pp. 95–100 (2005)

    Google Scholar 

  14. Fink, G.A., Wienecke, M., Sagerer, G.: Video-based on-line handwriting recognition. In: Proc. Int. Conf. on Document Analysis and Recognition, pp. 226–230. IEEE, Seattle (2001)

    Google Scholar 

  15. Fischer, A., Keller, A., Frinken, V., Bunke, H.: Lexicon-free handwritten word spotting using character hmms. Pattern Recognit. Lett. 33(7), 934–942 (2012)

    Article  Google Scholar 

  16. Grzeszick, R., Rothacker, L., Fink, G.A.: Bag-of-features representations using spatial visual vocabularies for object classification. In: IEEE Intl. Conf. on Image Processing, Melbourne, Australia (2013)

    Google Scholar 

  17. Hammerla, N.Y., Plötz, T., Vajda, S., Fink, G.A.: Towards feature learning for HMM-based offline handwriting recognition. In: International Workshop on Frontiers of Arabic Handwriting Recognition, Istanbul, Turkey (2010)

    Google Scholar 

  18. Kozielski, M., Doetsch, P., Ney, H.: Improvements in RWTH’s system for off-line handwriting recognition. In: Proc. Int. Conf. on Document Analysis and Recognition, Washington, DC, USA (2013)

    Google Scholar 

  19. Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proc. IEEE Comp. Soc. Conf. on Computer Vision and Pattern Recognition, vol. 2, pp. 2169–2178 (2006)

    Google Scholar 

  20. Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  21. Lu, Z., Schwartz, R., Raphael, C.: Script-independent, HMM-based text line finding for OCR. In: Proc. Int. Conf. on Pattern Recognition, Barcelona, vol. 4, pp. 551–554 (2000)

    Google Scholar 

  22. Madhvanath, S., Kim, G., Govindaraju, V.: Chaincode contour processing for handwritten word recognition. IEEE Trans. Pattern Anal. Mach. Intell. 21(9), 928–932 (1999)

    Article  Google Scholar 

  23. Marti, U.-V., Bunke, H.: Handwritten sentence recognition. In: Proc. Int. Conf. on Pattern Recognition, Barcelona, vol. 3, pp. 467–470 (2000)

    Google Scholar 

  24. Natarajan, P., Lu, Z., Schwartz, R., Bazzi, I., Makhoul, J.: Multilingual machine printed OCR. Int. J. Pattern Recognit. Artif. Intell. 15(1), 43–63 (2001)

    Article  Google Scholar 

  25. Natarajan, P., Saleem, S., Prasad, R., MacRostie, E., Subramanian, K.: Multi-lingual offline handwriting recognition using hidden Markov models: a script-independent approach. In: Doermann, D.S., Jaeger, S. (eds.) SACH 2006: Arabic and Chinese Handwriting Recognition. Lecture Notes in Computer Science, vol. 4768, pp. 231–250. Springer, Berlin (2008)

    Chapter  Google Scholar 

  26. O’Hara, S., Draper, B.A.: Introduction to the bag of features paradigm for image classification and retrieval. Comput. Res. Repository (2011). arXiv:1101.3354v1

  27. Plötz, T., Fink, G.A.: Markov Models for Handwriting Recognition. Springer Briefs in Computer Science. Springer, Berlin (2011)

    Book  MATH  Google Scholar 

  28. Plötz, T., Thurau, C., Fink, G.A.: Camera-based whiteboard reading: new approaches to a challenging task. In: Proc. Int. Conf. on Frontiers in Handwriting Recognition, Montreal, Canada, pp. 385–390 (2008)

    Google Scholar 

  29. Prasad, R., Saleem, S., Kamali, M., Meermeier, R., Natarajan, P.: Improvements in hidden Markov model based Arabic OCR. In: Proc. Int. Conf. on Pattern Recognition, pp. 1–4 (2008)

    Google Scholar 

  30. Rothacker, L., Fink, G.A., Banerjee, P., Bhattacharya, U., Chaudhuri, B.B.: Bag-of-features hmms for segmentation-free bangla word spotting. In: International Workshop on Multilingual OCR (MOCR), Washington DC, USA (2013)

    Google Scholar 

  31. Rothacker, L., Rusinol, M., Fink, G.A.: Bag-of-features HMMs for segmentation-free word spotting in handwritten documents. In: Proc. Int. Conf. on Document Analysis and Recognition, Washington DC, USA (2013)

    Google Scholar 

  32. Rothacker, L., Vajda, S., Fink, G.A.: Bag-of-features representations for offline handwriting recognition applied to Arabic script. In: Proc. Int. Conf. on Frontiers in Handwriting Recognition, Bari, Italy (2012)

    Google Scholar 

  33. Rusiñol, M., Aldavert, D., Toledo, R., Llados, J.: Browsing heterogeneous document collections by a segmentation-free word spotting method. In: Proc. Int. Conf. on Document Analysis and Recognition, Bejing, China, pp. 63–67 (2011)

    Google Scholar 

  34. Saleem, S., Cao, H., Subramanian, K., Kamali, M., Prasad, R., Natarajan, P.: Improvements in bbn’s hmm-based offline Arabic handwriting recognition system. In: Proc. Int. Conf. on Document Analysis and Recognition, pp. 773–777 (2009)

    Google Scholar 

  35. Salton, G., McGill, J.M.: Introduction to Modern Information Retrieval. McGraw-Hill, New York (1983)

    MATH  Google Scholar 

  36. Schwartz, R., LaPre, C., Makhoul, J., Raphael, C., Zhao, Y.: Language-independent OCR using a continuous speech recognition system. In: Proc. Int. Conf. on Pattern Recognition, Vienna, Austria, vol. 3, pp. 99–103 (1996)

    Chapter  Google Scholar 

  37. Sivic, J., Zisserman, A.: Video Google: a text retrieval approach to object matching in videos. In: Proc. Int. Conf. on Computer Vision, vol. 2, pp. 1470–1477 (2003)

    Chapter  Google Scholar 

  38. Wienecke, M., Fink, G.A., Sagerer, G.: Towards automatic video-based whiteboard reading. In: Proc. Int. Conf. on Document Analysis and Recognition, Edinburgh, Scotland, pp. 87–91 (2003)

    Google Scholar 

  39. Wienecke, M., Fink, G.A., Sagerer, G.: Toward automatic video-based whiteboard reading. Int. J. Doc. Anal. Recognit. 7(2–3), 188–200 (2005)

    Article  Google Scholar 

  40. Zhang, Z., Tan, C.L.: Restoration of images scanned from thick bound documents. In: Int. Conf. on Image Processing, Thessaloniki, Greece, October 2001, pp. 1074–1077 (2001)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, TU Dortmund University, Dortmund, Germany

    Gernot A. Fink

Authors
  1. Gernot A. Fink
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag London

About this chapter

Cite this chapter

Fink, G.A. (2014). Handwriting Recognition. In: Markov Models for Pattern Recognition. Advances in Computer Vision and Pattern Recognition. Springer, London. https://doi.org/10.1007/978-1-4471-6308-4_14

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-6308-4_14

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-6307-7

  • Online ISBN: 978-1-4471-6308-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation