Skip to main content
SpringerLink
Log in
Book cover

Machine Learning in Document Analysis and Recognition pp 139–161Cite as

Classification and Learning Methods for Character Recognition: Advances and Remaining Problems

  • Chapter

Part of the book series: Studies in Computational Intelligence ((SCI,volume 90))

Pattern classification methods based on learning-from-examples have been widely applied to character recognition from the 1990s and have brought forth significant improvements of recognition accuracies. This kind of methods include statistical methods, artificial neural networks, support vector machines, multiple classifier combination, etc. In this chapter, we briefly review the learning-based classification methods that have been successfully applied to character recognition, with a special section devoted to the classification of large category set. We then discuss the characteristics of these methods, and discuss the remaining problems in character recognition that can be potentially solved by machine learning methods.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Mori, S., Suen, C.Y., Yamamoto, K.: Historical review of OCR research and development. Proc. IEEE 80(7) (1992) 1029-1058

    Article  Google Scholar 

  2. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11) (1998) 2278-2324

    Article  Google Scholar 

  3. Suen, C.Y., Liu, K., Strathy, N.W.: Sorting and recognizing cheques and fi-nancial documents. In: Lee SW, Nakano Y (eds) Document Analysis Systems: Theory and Practice. Springer, LNCS 1655 (1999) 173-187

    Google Scholar 

  4. Liu, C.L., Nakashima, K., Sako, H., Fujisawa, H.: Handwritten digit recognition: Benchmarking of state-of-the-art techniques. Pattern Recognition 36(10) (2003) 2271-2285

    Article  MATH  Google Scholar 

  5. Marinai, S., Gori, M., Soda, G.: Artificial neural networks for document analysis and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 27(1) (2005) 23-35

    Article  Google Scholar 

  6. Jain, A.K., Duin, R.P.W., Mao, J.: Statistical pattern recognition: A review. IEEE Trans. Pattern Anal. Mach. Intell. 22(1) (2000) 4-37

    Article  Google Scholar 

  7. Fukunaga, K.: Introduction to Statistical Pattern Recognition. Academic Press, 2nd edition (1990)

    Google Scholar 

  8. Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press (1995)

    Google Scholar 

  9. Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification. Wiley Interscience, 2nd edition (2001)

    Google Scholar 

  10. Friedman, J.H.: Regularized discriminant analysis. J. Am. Statist. Ass. 84(405) (1989) 165-175

    Article  Google Scholar 

  11. Kimura, F., Takashina, K., Tsuruoka, S., Miyake, Y.: Modified quadratic dis-criminant functions and the application to Chinese character recognition. IEEE Trans. Pattern Anal. Mach. Intell. 9(1) (1987) 149-153

    Article  Google Scholar 

  12. Kimura, F., Wakabayashi, T., Tsuruoka, S., Miyake, Y.: Improvement of hand-written Japanese character recognition using weighted direction code histogram. Pattern Recognition 30(8) (1997) 1329-1337

    Article  Google Scholar 

  13. Ikeda, M., Tanaka, H., Motooka, T.: Projection distance method of recognition of handwritten characters. Trans. IPS Japan 24(1) (1983) 106-112

    Google Scholar 

  14. Nakajima, T., Wakabayashi, T., Kimura, F., Miyake, Y.: Accuracy improve- ment by compound discriminant functions for resembling character recognition. Trans. IEICE Japan J83-D-II(2) (2000) 623-633

    Google Scholar 

  15. Hinton, G.E., Dayan, P., Revow, M.: Modeling the manifolds of images of hand-written digits. IEEE Trans. Neural Networks 8(1) (1997) 65-74

    Article  Google Scholar 

  16. Kim, H.C., Kim, D., Bang, S.Y.: A numeral character recognition using the PCA mixture model. Pattern Recognition Letters 23 (2002) 103-111

    Article  MATH  Google Scholar 

  17. Tsay, M.K., Shyu, K.H., Chang, P.C.: Feature transformation with generalized LVQ for handwritten Chinese character recognition. IEICE Trans. Information and Systems E82-D(3) (1999) 687-92

    Google Scholar 

  18. Zhang, P., Bui, T., Suen, C.Y.: Hybrid feature extraction and feature selec- tion for improving recognition accuracy of handwritten numerals. In: Proc. 8th ICDAR, Seoul, Korea, 1 (2005) 136-140

    Google Scholar 

  19. Kawatani, T., Shimizu, H.: Handwritten Kanji recognition with the LDA method. In: Proc. 14th ICPR, Brisbane, 2 (1998) 1031-1035

    Google Scholar 

  20. Wakabayashi, T., Shi, M., Ohyama, W., Kimura, F.: Accuracy improvement of handwritten numeral recognition by mirror image learning. In: Proc. 6th ICDAR, Seattle (2001) 338-343

    Google Scholar 

  21. Simard, P.Y., Steinkraus, D., Platt, J.C.: Best practices for convolutional neural networks applied to visual document analysis. In: Proc. 7th ICDAR, Edinburgh, UK, 2 (2003) 958-962

    Google Scholar 

  22. Oh, I.S., Suen, C.Y.: A class-modular feedforward neural network for handwrit-ing recognition. Pattern Recognition 35(1) (2002) 229-244

    Article  MATH  Google Scholar 

  23. Pao, Y.H.: Adaptive Pattern Recognition and Neural Networks. Addison- Wesley, MA (1989)

    MATH  Google Scholar 

  24. Schürmann, J.: Pattern Classification: A Unified View of Statistical and Neural Approaches. Wiley Interscience (1996)

    Google Scholar 

  25. Kreßel, U., Schürmann, J.: Pattern classification techniques based on function approximation. In: Bunke H, Wang PSP (eds) Handbook of Character Recog-nition and Document Image Analysis, World Scientific (1997) 49-78

    Google Scholar 

  26. Franke, J.: Isolated handprinted digit recognition. In: Bunke H, Wang PSP (eds) Handbook of Character Recognition and Document Image Analysis, World Scientific (1997) 103-121

    Google Scholar 

  27. Liu, C.L., Sako, H.: Class-specific feature polynomial classifier for pattern classi- fication and its application to handwritten numeral recognition. Pattern Recognition 39(4) (2006) 669-681

    Article  MATH  Google Scholar 

  28. Kimura, F., Inoue, S., Wakabayashi, T., Tsuruoka, S., Miyake, Y.: Handwritten numeral recognition using autoassociative neural networks.In: Proc. 14th ICPR, Brisbane, 1 (1998) 166-171

    Google Scholar 

  29. Zhang, B., Fu, M., Yang, H.: A nonlinear neural network model of mixture of lo-cal principal component analysis: Application to handwritten digits recognition. Pattern Recognition 34(2) (2001) 203-214

    Article  MATH  Google Scholar 

  30. Kohonen, T.: The self-organizing map. Proc. IEEE 78(9) (1990) 1464-1480

    Article  Google Scholar 

  31. Liu, C.L., Nakagawa, M.: Evaluation of prototype learning algorithms for nearest neighbor classifier in application to handwritten character recognition. Pattern Recognition 34(3) (2001) 601-615

    Article  MATH  Google Scholar 

  32. Liu, C.L., Sako, H., Fujisawa, H.: Discriminative learning quadratic discriminant function for handwriting recognition. IEEE Trans. Neural Networks 15(2) (2004) 430-444

    Article  Google Scholar 

  33. Vapnik, V.: The Nature of Statistical Learning Theory. Springer-Verlag, New Work (1995)

    MATH  Google Scholar 

  34. Burges, C.J.C.: A tutorial on support vector machines for pattern recognition. Knowledge Discovery and Data Mining 2(2) (1998) 1-43

    Google Scholar 

  35. Kressel, U.: Pairwise classification and support vector machines. In: Schölkopf B, Burges CJC, Smola AJ (eds) Advances in Kernel Methods: Support Vector Learning, MIT Press (1999) 255-268

    Google Scholar 

  36. Bellili, A., Gilloux, M., Gallinari, P.: An MLP-SVM combination architecture for offline handwritten digit recognition: Reduction of recognition errors by support vector machines rejection mechanisms. Int. J. Document Analysis and Recogni-tion 5(4) (2003) 244-252

    Google Scholar 

  37. Dong, J.X., Krzyzak, A., Suen, C.Y.: An improved handwritten Chinese char- acter recognition system using support vector machine. Pattern Recognition Letters 26(12) (2005) 1849-1856

    Article  Google Scholar 

  38. Xu, L., Krzyzak, A., Suen, C.Y.: Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Trans. System Man Cyber-net. 22(3) (1992) 418-435

    Article  Google Scholar 

  39. Ho, T.K., Hull, J., Srihari, S.N.: Decision combination in multiple classifier systems. IEEE Trans. Pattern Anal. Mach. Intell. 16(1) (1994) 66-75

    Google Scholar 

  40. Rahman, A.F.R., Fairhurst, M.C.: Multiple classifier decision combination strategies for character recognition: A review. Int. J. Document Analysis and Recognition 5(4) (2003) 166-194

    Article  Google Scholar 

  41. Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20(3) (1998) 226-239

    Article  Google Scholar 

  42. Duin, R.P.W.: The combining classifiers: To train or not to train. Proc. 16th ICPR, Quebec, Canada, 2 (2002) 765-770

    Google Scholar 

  43. Liu, C.L.: Classifier combination based on confidence transformation. Pattern Recognition, 38(1) (2005) 11-28

    MATH  Google Scholar 

  44. Suen, C.Y., Lam, L.: Multiple classifier combination methodologies for different output levels. In: Kittler J, Roli F (eds) Multiple Classifier Systems, Springer, LNCS 1857 (2000) 52-66

    Google Scholar 

  45. Breiman, L.: Bagging predictors. Machine Learning 24(2) (1996) 123-140

    MATH  MathSciNet  Google Scholar 

  46. Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learn-ing and an application to boosting. J. Computer and System Sciences 55(1) (1997) 119-139

    Article  MATH  MathSciNet  Google Scholar 

  47. Ha, T., Bunke, H.: Off-line handwritten numeral recognition by perturbation method. IEEE Trans. Pattern Anal. Mach. Intell. 19(5) (1997) 535-539

    Article  Google Scholar 

  48. Dahmen, J., Keysers, D., Ney, H.: Combined classification of handwritten digits using the virtual test sample method. In : Kittler J, Roli F (eds) Multiple Classifier Systems, Springer, LNCS 2096 (2001) 99-108

    Google Scholar 

  49. Tang, Y.Y., et al.: Offline recognition of Chinese handwriting by multifeature and multilevel classification. IEEE Trans. Pattern Anal. Mach. Intell. 20(5) (1998) 556-561

    Article  Google Scholar 

  50. Wang, Q.R., Suen, C.Y.: Analysis and design of a decision tree based on entropy reduction and its application to large character set recognition. IEEE Trans. Pattern Anal. Mach. Intell. 6(4) (1984) 406-417

    Article  MATH  Google Scholar 

  51. Suzuki, M., Omachi, S., Kato, N., Aso, H., Nemoto, Y.: A discrimination method of similar characters using compound Mahalanobis function. Trans. IE-ICE Japan J80-D-II(10) (1997) 2752-2760

    Google Scholar 

  52. Kato, N., Suzuki, M., Omachi, S., Aso, H., Nemoto, Y.: A handwritten charac-ter recognition system using directional element feature and asymmetric Maha-lanobis distance. IEEE Trans. Pattern Anal. Mach. Intell. 21(3) (1999) 258-262

    Article  Google Scholar 

  53. Fu, H.C., Xu, Y.Y.: Multilinguistic handwritten character recognition by Bayesian decision-based neural networks. IEEE Trans. Signal Processing 46(10) (1998) 2781-2789

    Google Scholar 

  54. Kimura, Y., Wakahara, T., Tomono, A.: Combination of statistical and neural classifiers for a high-accuracy recognition of large character sets. Trans. IEICE Japan J83-D-II(10) (2000) 1986-1994

    Google Scholar 

  55. Saruta, K., Kato, N., Abe, M., Nemoto, Y.: High accuracy recognition of ETL9B using exclusive learning neural network-II (ELNET-II). IEICE Trans. Informa-tion and Systems 79-D(5) (1996) 516-521

    Google Scholar 

  56. Fukumoto, T., Wakabayashi, T., Kumura, F., Miyake, Y.: Accuracy improve-ment of handwritten character recognition by GLVQ. In: Proc. 7th IWFHR, Amsterdam (2000) 271-280

    Google Scholar 

  57. Liu, C.L.: High accuracy handwritten Chinese character recognition using quadratic classifiers with discriminative feature extraction. In: Proc. 18th ICPR, Hong Kong, 2 (2006) 942-945

    Google Scholar 

  58. Liu, H., Ding, X.: Handwritten character recognition using gradient feature and quadratic classifier with multiple discrimination schemes. In: Proc. 8th ICDAR, Seoul, Korea, 1 (2005) 19-23

    Google Scholar 

  59. Liu, H., Ding, X.: Handwritten Chinese character recognition based on mirror image learning and the compound Mahalanobis distance. J. Tsinghua Univ. (Sci & Tech) 46(7) (2006) 1239-1242 (in Chinese)

    Google Scholar 

  60. Liu, H., Ding, X.: Improve handwritten character recognition performance by heteroscedastic linear discriminant analysis, In: Proc. 18th ICPR, Hong Kong, 1(2006) 880-883

    Google Scholar 

  61. Biem, A., Katagiri, S., Juang, B.H.: Pattern recognition using discriminative feature extraction. IEEE Trans. Signal Processing 45(2) (1997) 500-504

    Article  Google Scholar 

  62. Huo, Q., Ge, Y., Feng, Z.D.: High performance Chinese OCR based on Ga-bor features, discriminative feature extraction and model training. In: Proc. ICASSP’01, Salt Lake City, Utah, 3 (2001) 1517-1520

    Google Scholar 

  63. Liu, C.L., Mine, R., Koga, M.: Building compact classifier for large character set recognition using discriminative feature extraction. In: Proc. 8th ICDAR, Seoul, Korea (2005) 846-850

    Google Scholar 

  64. Suen, C.Y., Nadal, C., Legault, R., Mai, T.A., Lam, L.: Computer recognition of unconstrained handwritten numerals. Proc. IEEE 80(7) (1982) 1162-1180

    Google Scholar 

  65. Liu, C.L., Nakashima, K., Sako, H., Fujisawa, H.: Handwritten digit recogni-tion: investigation of normalization and feature extraction techniques. Pattern Recognition 37(2) (2004) 265-279

    Article  MATH  Google Scholar 

  66. Teow, L.N., Loe, K.F.: Robust vision-based features and classification schemes for off-line handwritten digit recognition. Pattern Recognition 35(11) (2002) 2355-2364

    Article  MATH  Google Scholar 

  67. Holmström, L., Koistinen, P., Laaksonen, J., Oja, E.: Neural and statistical classifiers—taxonomy and two case studies. IEEE Trans. Neural Networks 8(1) (1997) 5-17

    Article  Google Scholar 

  68. Liu, C.L., Sako, H., Fujisawa, H.: Performance evaluation of pattern classifiers for handwritten character recognition. Int. J. Document Analysis and Recogni-tion 4(3) (2002) 191-204

    Article  Google Scholar 

  69. Tsukumo, J., Tanaka, H.: Classification of handprinted Chinese characters using non-linear normalization and correlation methods. In: Proc. 9th ICPR, Rome (1988) 168-171

    Google Scholar 

  70. Yamada, H., Yamamoto, K., Saito, T.: A nonlinear normalization method for hanprinted Kanji character recognition—line density equalization. Pattern Recognition 23(9) (1990) 1023-1029

    Article  Google Scholar 

  71. Gori, M., Scarselli, F.: Are multilayer perceptrons adequate for pattern recog-nition and verification? IEEE Trans. Pattern Anal. Mach. Intell. 20(11) (1998) 1121-1132

    Article  Google Scholar 

  72. Liu, C.L., Sako, H., Fujisawa, H.: Effects of classifier structures and training regimes on integrated segmentation and recognition of handwritten numeral strings. IEEE Trans. Pattern Anal. Mach. Intell. 26(11) (2004) 1395-1407

    Google Scholar 

  73. Liu, C.L., Marukawa, K.: Handwritten numeral string recognition: Character-level training vs. string-level training. In: Proc. 17th ICPR, Cambridge, UK, 1 (2004) 405-408

    Google Scholar 

  74. Raina, R., Shen, Y., Ng, A.Y., McCallum, A.: Classification with hybrid gen-erative/discriminative models. In: Advances in Neural Information Processing System 16 (2003)

    Google Scholar 

  75. Dahmen, J., Schluter, R., Ney, H.: Discriminative training of Gaussian mixtures for image object recognition. In: Proc. 21st Symposium of German Association for Pattern Recognition, Bonn, Germany (1999) 205-212

    Google Scholar 

  76. Tax, D.M.J., Duin, R.P.W.: Support vector data description. Machine Learning 54(1)(2004) 45-66

    Article  MATH  Google Scholar 

  77. Suen, C.Y., Tan, J.: Analysis of error of handwritten digits made by a multitude of classifiers. Pattern Recognition Letters 26(3) (2005) 369-379

    Article  Google Scholar 

  78. Polikar, R., Udpa, L., Udpa, A.S., Honavar, V.: Learn++: An incremental learn-ing algorithm for supervised neural networks. IEEE Trans. System Man Cyber-net. Part C 31(4) (2001) 497-508

    Article  Google Scholar 

  79. Chawla, N.V., Karakoulas, G.: Learning from labeled and unlabeled data: An empirical study across techniques and domains. J. Artificial Intelligence Re-search 23 (2005) 331-366

    MATH  Google Scholar 

  80. Duin, R.P.W.: A note on comparing classifiers. Pattern Recognition Letters, 17 (1996) 529-536

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences, 2728, Beijing, 100080, P.R. China

    Cheng-Lin Liu

  2. Central Research Laboratory, Hitachi Ltd., 1-280 Higashi-koigakubo, Kokubunji-shi, Tokyo, 185-8601, Japan

    Hiromichi Fujisawa

Authors
  1. Cheng-Lin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hiromichi Fujisawa
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Dipartimento di Sistemi e Informatica, University of Florence, Via S. Marta, 3, 50139, Firenze, Italy

    Simone Marinai

  2. Hitachi Central Research Laboratory, 1-280, Higashi-Koigakubo, Kokubunji-shi, Tokyo, 185-8601, Japan

    Hiromichi Fujisawa

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Liu, CL., Fujisawa, H. (2008). Classification and Learning Methods for Character Recognition: Advances and Remaining Problems. In: Marinai, S., Fujisawa, H. (eds) Machine Learning in Document Analysis and Recognition. Studies in Computational Intelligence, vol 90. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-76280-5_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-76280-5_6

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-76279-9

  • Online ISBN: 978-3-540-76280-5

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation