A Performance Characterization Algorithm for Symbol Localization

  • Mathieu Delalandre
  • Jean-Yves Ramel
  • Ernest Valveny
  • Muhammad Muzzamil Luqman
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6020)


In this paper we present an algorithm for performance characterization of symbol localization systems. This algorithm is aimed to be a more “reliable” and “open” solution to characterize the performance. To achieve that, it exploits only single points as the result of localization and offers the possibility to reconsider the localization results provided by a system. We use the information about context in groundtruth, and overall localization results, to detect the ambiguous localization results. A probability score is computed for each matching between a localization point and a groundtruth region, depending on the spatial distribution of the other regions in the groundtruth. Final characterization is given with detection rate/probability score plots, describing the sets of possible interpretations of the localization results, according to a given confidence rate. We present experimentation details along with the results for the symbol localization system of [1], exploiting a synthetic dataset of architectural floorplans and electrical diagrams (composed of 200 images and 3861 symbols).


symbol localization groundtruth performance characterization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Qureshi, R., Ramel, J., Barret, D., Cardot, H.: Symbol spotting in graphical documents using graph representations. In: Liu, W., Lladós, J., Ogier, J.-M. (eds.) GREC 2007. LNCS, vol. 5046, pp. 91–103. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  2. 2.
    Valveny, E., Tabbone, S., Ramos, O., Philippot, E.: Performance characterization of shape descriptors for symbol representation. In: Liu, W., Lladós, J., Ogier, J.-M. (eds.) GREC 2007. LNCS, vol. 5046, pp. 278–287. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  3. 3.
    Delalandre, M., Valveny, E., Lladós, J.: Performance evaluation of symbol recognition and spotting systems: An overview. In: Workshop on Document Analysis Systems (DAS), pp. 497–505 (2008)Google Scholar
  4. 4.
    Rusiñol, M., Lladós, J.: A performance evaluation protocol for symbol spotting systems in terms of recognition and location indices. International Journal on Document Analysis and Recognition (IJDAR) 12(2), 83–96 (2009)Google Scholar
  5. 5.
    Delalandre, M., Pridmore, T., Valveny, E., Trupin, E., Locteau, H.: Building synthetic graphical documents for performance evaluation. In: Liu, W., Lladós, J., Ogier, J.-M. (eds.) GREC 2007. LNCS, vol. 5046, pp. 288–298. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  6. 6.
    Locteau, H., Adam, S., Trupin, E., Labiche, J., Heroux, P.: Symbol spotting using full visibility graph representation. In: Liu, W., Lladós, J., Ogier, J.-M. (eds.) GREC 2007. LNCS, vol. 5046, pp. 49–50. Springer, Heidelberg (2008)Google Scholar
  7. 7.
    Unnikrishnan, R., Pantofaru, C., Hebert, M.: Toward objective evaluation of image segmentation algorithms. Pattern Analysis and Machine Intelligence (PAMI) 29(6), 929–944 (2007)CrossRefGoogle Scholar
  8. 8.
    Breuel, T.: Representations and metrics for off-line handwriting segmentation. In: International Workshop on Frontiers in Handwriting Recognition (IWFHR), pp. 428–433 (2002)Google Scholar
  9. 9.
    Bridson, D., Antonacopoulos, A.: A geometric approach for accurate and efficient performance evaluation. In: International Conference on Pattern Recognition (ICPR), pp. 1–4 (2008)Google Scholar
  10. 10.
    Wenyin, L., Dori, D.: A proposed scheme for performance evaluation of graphics/text separation algorithms. In: Chhabra, A.K., Tombre, K. (eds.) GREC 1997. LNCS, vol. 1389, pp. 335–346. Springer, Heidelberg (1998)Google Scholar
  11. 11.
    Kanungo, T., Resnik, P.: The bible, truth, and multilingual ocr evaluation. In: Document Recognition and Retrieval (DRR). SPIE Proceedings, vol. 3651, pp. 86–96 (1999)Google Scholar
  12. 12.
    Balaban, I.: An optimal algorithm for finding segments intersections. In: Symposium on Computational Geometry (SGC), pp. 211–219 (1995)Google Scholar
  13. 13.
    Kanungo, T., Haralick, R.M., Phillips, I.: Non-linear local and global document degradation models. International Journal of Imaging Systems and Technology (IJIST) 5(3), 220–230 (1994)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Mathieu Delalandre
    • 1
    • 2
  • Jean-Yves Ramel
    • 1
  • Ernest Valveny
    • 2
  • Muhammad Muzzamil Luqman
    • 1
    • 2
  1. 1.Laboratoire d’Informatique (LI)ToursFrance
  2. 2.Computer Vision Center (CVC)Bellaterra (Barcelona)Spain

Personalised recommendations