Abstract

In contrast to the field of automatic speech recognition where Markov model-based methods currently represent the state-of-the-art, HMMs and n-gram models are still a rather new approach for the recognition of machine-printed or handwritten texts. In this chapter we will present state-of-the-art systems for offline handwriting recognition. In addition to explanations of how Markov model technology is applied, these presentations will also include brief descriptions of the specialized methods for preprocessing and feature extraction used.

The first system presented in this chapter is BBN’s offline HWR system. It can be considered as a typical example for an HMM-based system for the recognition of machine-printed or handwritten script. Afterwards, we will present the offline HWR system of RWTH Aachen University, Aachen, Germany. The chapter concludes with a presentation of our own systems for offline handwriting recognition which include a system based on so-called Bag-of-Features HMMs. This recently proposed extension of the HMM framework works especially well on the challenging task of query-by-example word spotting.

Keywords

Covariance Pyramid 

References

  1. 13.
    Basha Shaik, M.A., Rybach, D., Hahn, S., Schlüter, R., Ney, H.: Hierarchical hybrid language models for open vocabulary continuous speech recognition using wfst. In: Proc. Workshop on Statistical and Perceptual Audition, Portland, OR, USA, pp. 46–51 (2012) Google Scholar
  2. 16.
    Bazzi, I., Schwartz, R., Makhoul, J.: An omnifont open-vocabulary OCR system for English and Arabic. IEEE Trans. on Pattern Analysis and Machine Intelligence 21(6), 495–504 (1999) CrossRefGoogle Scholar
  3. 38.
    Caesar, T., Gloger, J.M., Mandler, E.: Preprocessing and feature extraction for a handwriting recognition system. In: Proc. Int. Conf. on Document Analysis and Recognition, Tsukuba Science City, Japan, pp. 408–411 (1993) Google Scholar
  4. 49.
    Colthurst, T., Kimball, O., Richardson, F., Shu, H., Wooters, C., Iyer, R., Gish, H.: The 2000 BBN Byblos LVCSR system. In: 2000 Speech Transcription Workshop, Maryland (2000) Google Scholar
  5. 60.
    Dolfing, J.G.A., Haeb-Umbach, R.: Signal representations for Hidden Markov Model based on-line handwriting recognition. In: Proc. Int. Conf. on Acoustics, Speech, and Signal Processing, München, vol. IV, pp. 3385–3388 (1997) Google Scholar
  6. 61.
    Dreuw, P., Jonas, S., Ney, H.: White-space models for offline Arabic handwriting recognition. In: Proc. Int. Conf. on Pattern Recognition, Tampa, FL, USA, pp. 1–4 (2008) Google Scholar
  7. 63.
    Dreuw, P., Rybach, D., Heigold, G., Ney, H.: Rwth ocr: a large vocabulary optical character recognition system for Arabic scripts. In: Märgner, V., El Abed, H. (eds.) Guide to OCR for Arabic Scripts, pp. 215–254. Springer, London, UK (2012). Chap. Part II: Recognition CrossRefGoogle Scholar
  8. 78.
    Fei-Fei, L., Perona, P.: A Bayesian hierarchical model for learning natural scene categories. In: Proc. IEEE Comp. Soc. Conf. on Computer Vision and Pattern Recognition, vol. 2, pp. 524–531 (2005) Google Scholar
  9. 83.
    Fink, G.A., Plötz, T.: On appearance-based feature extraction methods for writer-independent handwritten text recognition. In: Proc. Int. Conf. on Document Analysis and Recognition, Seoul, Korea, vol. 2, pp. 1070–1074 (2005) Google Scholar
  10. 84.
    Fink, G.A., Plötz, T.: Unsupervised estimation of writing style models for improved unconstrained off-line handwriting recognition. In: Proc. Int. Workshop on Frontiers in Handwriting Recognition, La Baule, France, pp. 429–434 (2006) Google Scholar
  11. 87.
    Fink, G.A., Plötz, T.: On the use of context-dependent modelling units for HMM-based offline handwriting recognition. In: Proc. Int. Conf. on Document Analysis and Recognition, Curitiba, Brazil, vol. 2, pp. 729–733 (2007) Google Scholar
  12. 90.
    Fink, G.A., Vajda, S., Bhattacharya, U., Parui, S.K., Chaudhuri, B.B.: Online Bangla word recognition using sub-stroke level features and hidden Markov models. In: Proc. Int. Conf. on Frontiers in Handwriting Recognition, Kolkata, India, pp. 393–398 (2010) Google Scholar
  13. 91.
    Fink, G.A., Wienecke, M.: Experiments in video-based whiteboard reading. In: First Int. Workshop on Camera-Based Document Analysis and Recognition, Seoul, Korea, pp. 95–100 (2005) Google Scholar
  14. 93.
    Fink, G.A., Wienecke, M., Sagerer, G.: Video-based on-line handwriting recognition. In: Proc. Int. Conf. on Document Analysis and Recognition, pp. 226–230. IEEE, Seattle (2001) Google Scholar
  15. 95.
    Fischer, A., Keller, A., Frinken, V., Bunke, H.: Lexicon-free handwritten word spotting using character hmms. Pattern Recognit. Lett. 33(7), 934–942 (2012) CrossRefGoogle Scholar
  16. 113.
    Grzeszick, R., Rothacker, L., Fink, G.A.: Bag-of-features representations using spatial visual vocabularies for object classification. In: IEEE Intl. Conf. on Image Processing, Melbourne, Australia (2013) Google Scholar
  17. 117.
    Hammerla, N.Y., Plötz, T., Vajda, S., Fink, G.A.: Towards feature learning for HMM-based offline handwriting recognition. In: International Workshop on Frontiers of Arabic Handwriting Recognition, Istanbul, Turkey (2010) Google Scholar
  18. 159.
    Kozielski, M., Doetsch, P., Ney, H.: Improvements in RWTH’s system for off-line handwriting recognition. In: Proc. Int. Conf. on Document Analysis and Recognition, Washington, DC, USA (2013) Google Scholar
  19. 167.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proc. IEEE Comp. Soc. Conf. on Computer Vision and Pattern Recognition, vol. 2, pp. 2169–2178 (2006) Google Scholar
  20. 183.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004) CrossRefGoogle Scholar
  21. 186.
    Lu, Z., Schwartz, R., Raphael, C.: Script-independent, HMM-based text line finding for OCR. In: Proc. Int. Conf. on Pattern Recognition, Barcelona, vol. 4, pp. 551–554 (2000) Google Scholar
  22. 188.
    Madhvanath, S., Kim, G., Govindaraju, V.: Chaincode contour processing for handwritten word recognition. IEEE Trans. Pattern Anal. Mach. Intell. 21(9), 928–932 (1999) CrossRefGoogle Scholar
  23. 191.
    Marti, U.-V., Bunke, H.: Handwritten sentence recognition. In: Proc. Int. Conf. on Pattern Recognition, Barcelona, vol. 3, pp. 467–470 (2000) Google Scholar
  24. 203.
    Natarajan, P., Lu, Z., Schwartz, R., Bazzi, I., Makhoul, J.: Multilingual machine printed OCR. Int. J. Pattern Recognit. Artif. Intell. 15(1), 43–63 (2001) CrossRefGoogle Scholar
  25. 204.
    Natarajan, P., Saleem, S., Prasad, R., MacRostie, E., Subramanian, K.: Multi-lingual offline handwriting recognition using hidden Markov models: a script-independent approach. In: Doermann, D.S., Jaeger, S. (eds.) SACH 2006: Arabic and Chinese Handwriting Recognition. Lecture Notes in Computer Science, vol. 4768, pp. 231–250. Springer, Berlin (2008) CrossRefGoogle Scholar
  26. 221.
    O’Hara, S., Draper, B.A.: Introduction to the bag of features paradigm for image classification and retrieval. Comput. Res. Repository (2011). arXiv:1101.3354v1
  27. 241.
    Plötz, T., Fink, G.A.: Markov Models for Handwriting Recognition. Springer Briefs in Computer Science. Springer, Berlin (2011) CrossRefMATHGoogle Scholar
  28. 243.
    Plötz, T., Thurau, C., Fink, G.A.: Camera-based whiteboard reading: new approaches to a challenging task. In: Proc. Int. Conf. on Frontiers in Handwriting Recognition, Montreal, Canada, pp. 385–390 (2008) Google Scholar
  29. 246.
    Prasad, R., Saleem, S., Kamali, M., Meermeier, R., Natarajan, P.: Improvements in hidden Markov model based Arabic OCR. In: Proc. Int. Conf. on Pattern Recognition, pp. 1–4 (2008) Google Scholar
  30. 257.
    Rothacker, L., Fink, G.A., Banerjee, P., Bhattacharya, U., Chaudhuri, B.B.: Bag-of-features hmms for segmentation-free bangla word spotting. In: International Workshop on Multilingual OCR (MOCR), Washington DC, USA (2013) Google Scholar
  31. 258.
    Rothacker, L., Rusinol, M., Fink, G.A.: Bag-of-features HMMs for segmentation-free word spotting in handwritten documents. In: Proc. Int. Conf. on Document Analysis and Recognition, Washington DC, USA (2013) Google Scholar
  32. 259.
    Rothacker, L., Vajda, S., Fink, G.A.: Bag-of-features representations for offline handwriting recognition applied to Arabic script. In: Proc. Int. Conf. on Frontiers in Handwriting Recognition, Bari, Italy (2012) Google Scholar
  33. 261.
    Rusiñol, M., Aldavert, D., Toledo, R., Llados, J.: Browsing heterogeneous document collections by a segmentation-free word spotting method. In: Proc. Int. Conf. on Document Analysis and Recognition, Bejing, China, pp. 63–67 (2011) Google Scholar
  34. 264.
    Saleem, S., Cao, H., Subramanian, K., Kamali, M., Prasad, R., Natarajan, P.: Improvements in bbn’s hmm-based offline Arabic handwriting recognition system. In: Proc. Int. Conf. on Document Analysis and Recognition, pp. 773–777 (2009) Google Scholar
  35. 265.
    Salton, G., McGill, J.M.: Introduction to Modern Information Retrieval. McGraw-Hill, New York (1983) MATHGoogle Scholar
  36. 280.
    Schwartz, R., LaPre, C., Makhoul, J., Raphael, C., Zhao, Y.: Language-independent OCR using a continuous speech recognition system. In: Proc. Int. Conf. on Pattern Recognition, Vienna, Austria, vol. 3, pp. 99–103 (1996) CrossRefGoogle Scholar
  37. 284.
    Sivic, J., Zisserman, A.: Video Google: a text retrieval approach to object matching in videos. In: Proc. Int. Conf. on Computer Vision, vol. 2, pp. 1470–1477 (2003) CrossRefGoogle Scholar
  38. 316.
    Wienecke, M., Fink, G.A., Sagerer, G.: Towards automatic video-based whiteboard reading. In: Proc. Int. Conf. on Document Analysis and Recognition, Edinburgh, Scotland, pp. 87–91 (2003) Google Scholar
  39. 317.
    Wienecke, M., Fink, G.A., Sagerer, G.: Toward automatic video-based whiteboard reading. Int. J. Doc. Anal. Recognit. 7(2–3), 188–200 (2005) CrossRefGoogle Scholar
  40. 330.
    Zhang, Z., Tan, C.L.: Restoration of images scanned from thick bound documents. In: Int. Conf. on Image Processing, Thessaloniki, Greece, October 2001, pp. 1074–1077 (2001) Google Scholar

Copyright information

© Springer-Verlag London 2014

Authors and Affiliations

  • Gernot A. Fink
    • 1
  1. 1.Department of Computer ScienceTU Dortmund UniversityDortmundGermany

Personalised recommendations