Abstract
Symbol recognition is one of the central problems in the field of graphics recognition. Many methods and approaches have been developed in the context of several application domains. In the last years, the need for generic methods, able to perform well on large sets of symbols in different domains, has become clear. Thus, standard evaluation datasets and protocols have to be built up in order to be able to evaluate the performance of all these methods. In this paper we discuss several points which should be taken into account in the design of such evaluation framework, raising a number of open questions for further discussion. These issues were the starting point of the organization of the contest on symbol recognition held during the last Workshop on Graphics Recognition (GREC’03). We also summarize the main features of the dataset and the protocol of evaluation used in the contest, as a first step to define a general evaluation framework, giving answer to these open questions.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Haralick, R.: Performance characterization in image analysis: Tninning, a case in point. Pattern Recognition Letters 13, 5–12 (1992)
Guyon, I., Haralick, R., Hull, J., Phipliops, I.: Data sets for ocr and document image understanding research. In: Bunke, H., Wang, P. (eds.) Handbook of Character Recognition and Document Image Analysis, pp. 779–800. World Scientific, Singapore (1997)
Kong, B., Phillips, I.T., Haralick, R.M., Prasad, A., Kasturi, R.: A benchmark: Performance evaluation of dashed-line detection algorithms. In: Kasturi, R., Tombre, K. (eds.) Graphics Recognition 1995. LNCS, vol. 1072, pp. 270–285. Springer, Heidelberg (1996)
Chhabra, A., Phillips, I.: The second international graphics recognition contest - raster to vector conversion: A report. In: [20], pp. 390–410
Chhabra, A., Philips, I.: Performance evaluation of line drawing recognition systems. In: Proceedings of 15th. International Conference on Pattern Recognition, Barcelona, Spain, vol. 4, pp. 864–869 (2000)
Wenyin, L., Zhai, J., Dori, D.: Extended summary of the arc segmentation contest. In: Blostein, D., Kwon, Y.-B. (eds.) GREC 2001. LNCS, vol. 2390, pp. 343–349. Springer, Heidelberg (2002)
Wenyin, L., Dori, D.: A protocol for performance evaluation of line detection algorithms. Machine Vision and Applications 9, 240–250 (1997)
Phillips, I., Chhabra, A.: Empirical performance evaluation of graphics recognition systems. IEEE Transactions on Pattern Analyisis and Machine Intelligence 21, 849–870 (1999)
Hilaire, X.: A matching scheme to enhance performance evaluation of rasterto- vector conversion algorithms. In: Proceedings of 7th International Conference on Document Analysis and Recognition, Edinburgh, Scotland, vol. 1, pp. 629–633 (2003)
Aksoy, S., Ye, M., Schauf, M., Song, M., Wang, Y., Haralick, R., Parker, J., Pivovarov, J., Royko, D., Sun, C., Farneboock, G.: Algorithm performance contest. In: Proceedings of 15th. International Conference on Pattern Recognition, Barcelona, Spain, vol. 4, pp. 870–876 (2000)
Valveny, E., Dosch, P.: Symbol recognition contest: A synthesis. In: Selected Papers from Fith International Workshop on Graphics Recognition, GREC 2003 (2004) (to appear)
Chhabra, A.K.: Graphic Symbol Recognition: An Overview. In: [20], pp. 68–79
Cordella, L.P., Vento, M.: Symbol recognition in documents: a collection of techniques? International Journal on Document Analysis and Recognition 3, 73–88 (2000)
Lladós, J., Valveny, E., Sánchez, G., Martí, E.: Symbol Recognition: Current Advances and Perspectives. In: Blostein, D., Kwon, Y.-B. (eds.) GREC 2001. LNCS, vol. 2390, pp. 104–127. Springer, Heidelberg (2002)
Tombre, K., Ah-Soon, C., Dosch, P., Masini, G., Tabbone, S.: Stable and Robust Vectorization: How to Make the Right Choices. In: Chhabra, A.K., Dori, D. (eds.) GREC 1999. LNCS, vol. 1941, pp. 3–18. Springer, Heidelberg (2000)
Baird, H.S.: The State of the Art of Document Image Degradation Modeling. In: Proceedings of 4th IAPR International Workshop on Document Analysis Systems, Rio de Janeiro (Brazil) (2000)
Kanungo, T., Haralick, R.M., Baird, H.S., Stuetzle, W., Madigan, D.: Document Degradation Models: Parameter Estimation and Model Validation. In: Proceedings of IAPR Workshop on Machine Vision Applications, Kawasaki (Japan), pp. 552–557 (1994)
Tombre, K., Chhabra, A.K.: General Conclusions. In: [20], pp. 411–420
Cootes, T., Taylor, C., Cooper, D., Graham, J.: Active shape models: Their training and application. Computer Vision and Image Understanding 61, 38–59 (1995)
Chhabra, A.K., Tombre, K. (eds.): GREC 1997. LNCS, vol. 1389. Springer, Heidelberg (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Valveny, E., Dosch, P. (2004). Performance Evaluation of Symbol Recognition. In: Marinai, S., Dengel, A.R. (eds) Document Analysis Systems VI. DAS 2004. Lecture Notes in Computer Science, vol 3163. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-28640-0_34
Download citation
DOI: https://doi.org/10.1007/978-3-540-28640-0_34
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23060-1
Online ISBN: 978-3-540-28640-0
eBook Packages: Springer Book Archive