A Rule-Based Approach to Form Mathematical Symbols in Printed Mathematical Expressions

  • P. Pavan Kumar
  • Arun Agarwal
  • Chakravarthy Bhagvati
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7080)


Automated understanding of mathematical expressions (MEs) is currently a challenging task due to their complex two- dimensional (2D) structure. Recognition of MEs can be online or offline and in either case, the process involves symbol recognition and analysis of 2D structure. This process is more complex for offline or printed MEs as they do not have temporal information. In our present work, we focus on the recognition of printed MEs and assume connected components (ccs) of a given ME image are labelled. Our approach to ME recognition comprises three stages,namely symbol formation, structural analysis and generation of encoding form like LATEX. In this paper, we present symbol formation process, where multi-cc symbols (like =, ≡ etc.) are formed, identity of context-dependent symbols (like a horizontal line can be MINUS, OVERBAR, FRACTION etc.) are resolved using spatial relations. Multi-line MEs like matrices and enumerated functions are also handled in this stage. A rule-based approach is proposed for the purpose, where the heuristics based on spatial relations are represented in the form of rules (knowledge) and those rules are fired depending on input data (labelled ccs). As knowledge is isolated from data like an expert system in our approach, it allows for easy adaptability and extensibility of the process. Proposed approach also handles both single-line and multi-line MEs in an unified manner. Our approach has been tested on around 800 MEs collected from various mathematical documents and experimental results are reported on them.


Mathematical expressions connected components symbol formation rule-based approach 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
  2. 2.
    flex++(1):fast lexical analyzer generator-Linux man page,
  3. 3.
    Awal, A.-M., Mouchere, H., Viard-Gaudin, C.: Towards handwritten mathematical expression recognition. In: ICDAR 2009, pp. 1046–1050. IEEE Computer Society, Washington, DC (2009)Google Scholar
  4. 4.
    Buchanan, B.G., Shortliffe, E.H.: Rule Based Expert Systems: The Mycin Experiments of the Stanford Heuristic Programming Project. Addison-Wesley (1984)Google Scholar
  5. 5.
    Chan, K.-F., Yeung, D.-Y.: Mathematical expression recognition: a survey. IJDAR 3, 3–15 (2000)CrossRefGoogle Scholar
  6. 6.
    Chaudhuri, B.B., Garain, U.: An approach for recognition and interpretation of mathematical expressions in printed document. Pattern Analysis and Applications 3, 120–131 (2000)CrossRefGoogle Scholar
  7. 7.
    Eto, Y., Suzuki, M.: Mathematical formula recognition using virtual link network. In: ICDAR 2001, pp. 762–767. IEEE Computer Society, Washington, DC (2001)Google Scholar
  8. 8.
    Fukuda, R., Sou, I., Tamari, F., Ming, X., Suzuki, M.: A technique of mathematical expression structure analysis for the handwriting input system. In: ICDAR 1999, p. 131. IEEE Computer Society, Washington, DC (1999)Google Scholar
  9. 9.
    Garain, U., Chaudhuri, B.B.: Recognition of online handwritten mathematical expressions. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 34(6), 2366–2376 (2004)CrossRefGoogle Scholar
  10. 10.
    Kanahori, T., Suzuki, M.: Detection of matrices and segmentation of matrix elements in scanned images of scientific documents. In: ICDAR 2003, vol. 1, p. 433. IEEE Computer Society, Washington, DC (2003)Google Scholar
  11. 11.
    Lee, H.J., Wang, J.S.: Design of a mathematical expression recognition system. In: ICDAR 1995, vol. 2, pp. 1084–1087 (1995)Google Scholar
  12. 12.
    Li, C., Zeleznik, R.C., Miller, T., LaViola, J.J.: Online recognition of handwritten mathematical expressions with support for matrices. In: ICPR 2008, pp. 1–4 (2008)Google Scholar
  13. 13.
    Suzuki, M., Tamari, F., Fukuda, R., Uchida, S., Kanahori, T.: Infty- an integrated OCR system for mathematical documents. In: Proceedings of ACM Symposium on Document Engineering 2003, pp. 95–104. ACM Press (2003)Google Scholar
  14. 14.
    Tapia, E., Rojas, R.: Recognition of on-line handwritten mathematical formulas in the E-Chalk System. In: ICDAR 2003, vol. 2, p. 980 (2003)Google Scholar
  15. 15.
    Tapia, E., Rojas, R.: Recognition of On-Line Handwritten Mathematical Expressions Using a Minimum Spanning Tree Construction and Symbol Dominance. In: Lladós, J., Kwon, Y.-B. (eds.) GREC 2003. LNCS, vol. 3088, pp. 329–340. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  16. 16.
    Tian, X.-D., Li, H.-Y., Li, X.-F., Zhang, L.-P.: Research on symbol recognition for mathematical expressions. In: International Conference on Innovative Computing, Information and Control, vol. 3, pp. 357–360. IEEE Computer Society, Los Alamitos (2006)Google Scholar
  17. 17.
    Tian, X., Fan, H.: Structural analysis based on baseline in printed mathematical expressions. In: International Conference on Parallel and Distributed Computing Applications and Technologies, PDCAT 2005, pp. 787–790. IEEE Computer Society, Los Alamitos (2005)Google Scholar
  18. 18.
    Toshihiro, K., Masakazu, S.: A Recognition Method of Matrices by Using Variable Block Pattern Elements Generating Rectangular Area. In: Blostein, D., Kwon, Y.-B. (eds.) GREC 2001. LNCS, vol. 2390, pp. 320–329. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  19. 19.
    Twaaliyondo, H.M., Okamoto, M.: Structure analysis and recognition of mathematical expressions. In: ICDAR 1995, vol. 1, p. 430. IEEE Computer Society, Washington, DC (1995)Google Scholar
  20. 20.
    Vuong, B.-Q., Hui, S.-C., He, Y.: Progressive structural analysis for dynamic recognition of on-line handwritten mathematical expressions. Pattern Recognition Letters 29, 647–655 (2008)CrossRefGoogle Scholar
  21. 21.
    Zanibbi, R., Blostein, D., Cordy, J.R.: Recognizing mathematical expressions using tree transformation. IEEE Transactions on PAMI 24, 1455–1467 (2002)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • P. Pavan Kumar
    • 1
  • Arun Agarwal
    • 1
  • Chakravarthy Bhagvati
    • 1
  1. 1.Dept. of Computer and Information SciencesUniversity of HyderabadIndia

Personalised recommendations