Object Reading: Text Recognition for Object Recognition

  • Sezer Karaoglu
  • Jan C. van Gemert
  • Theo Gevers
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7585)


We propose to use text recognition to aid in visual object class recognition. To this end we first propose a new algorithm for text detection in natural images. The proposed text detection is based on saliency cues and a context fusion step. The algorithm does not need any parameter tuning and can deal with varying imaging conditions. We evaluate three different tasks: 1. Scene text recognition, where we increase the state-of-the-art by 0.17 on the ICDAR 2003 dataset. 2. Saliency based object recognition, where we outperform other state-of-the-art saliency methods for object recognition on the PASCAL VOC 2011 dataset. 3. Object recognition with the aid of recognized text, where we are the first to report multi-modal results on the IMET set. Results show that text helps for object class recognition if the text is not uniquely coupled to individual object instances.


Object Recognition Optical Character Recognition Text Detection Text Recognition Scene Text 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Wang, K., Babenko, B., Belongie, S.: End-to-end scene text recognition. In: ICCV (2011)Google Scholar
  2. 2.
    Epshtein, B., Ofek, E., Wexler, Y.: Detecting text in natural scenes with stroke width transform. In: CVPR, pp. 2963–2970 (2010)Google Scholar
  3. 3.
    Neumann, L., Matas, J.: A Method for Text Localization and Recognition in Real-World Images. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part III. LNCS, vol. 6494, pp. 770–783. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  4. 4.
    Wang, K., Rescorla, E., Shacham, H., Belongie, S.: Openscan: A fully transparent optical scan voting system. In: Electronic Voting Technology Workshop (2010)Google Scholar
  5. 5.
    Judd, T., Ehinger, K.A., Durand, F., Torralba, A.: Learning to predict where humans look. In: ICCV, pp. 2106–2113 (2009)Google Scholar
  6. 6.
    Wang, H.C., Pomplun, M.: The attraction of visual attention to texts in real-world scenes. In: CogSci 2011 (2011)Google Scholar
  7. 7.
    Shahab, A., Shafait, F., Dengel, A., Uchida, S.: How salient is scene text? In: IAPR International Workshop on Document Analysis Systems (2012)Google Scholar
  8. 8.
    Zhu, Q., Yeh, M.C., Cheng, K.T.: Multimodal fusion using learned text concepts for image categorization. In: ACM MM (2006)Google Scholar
  9. 9.
    Uchida, S., Shigeyoshi, Y., Kunishige, Y., Feng, Y.: A keypoint-based approach toward scenery character detection. In: ICDAR, pp. 819–823 (2011)Google Scholar
  10. 10.
    van de Weijer, J., Gevers, T., Bagdanov, A.D.: Boosting color saliency in image feature detection. TPAMI 28, 150–156 (2006)CrossRefGoogle Scholar
  11. 11.
    Valenti, R., Sebe, N., Gevers, T.: Image saliency by isocentric curvedness and color. In: ICCV, pp. 2185–2192 (2009)Google Scholar
  12. 12.
    Zhang, J., Kasturi, R.: Text detection using edge gradient and graph spectrum. In: ICPR, pp. 3979–3982 (2010)Google Scholar
  13. 13.
    Neumann, L., Matas, J.: Text localization in real-world images using efficiently pruned exhaustive search. In: ICDAR, pp. 687–691 (2011)Google Scholar
  14. 14.
    Achanta, R., Hemami, S., Estrada, F., Susstrunk, S.: Frequency-tuned Salient Region Detection. In: CVPR (2009)Google Scholar
  15. 15.
    Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., Shum, H.Y.: Learning to detect a salient object. TPAMI 33 (2011)Google Scholar
  16. 16.
    van Gemert, J.C.: Exploiting photographic style for category-level image classification by generalizing the spatial pyramid. In: ICMR (2011)Google Scholar
  17. 17.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: CVPR (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Sezer Karaoglu
    • 1
  • Jan C. van Gemert
    • 1
  • Theo Gevers
    • 1
  1. 1.Intelligent Systems Lab Amsterdam (ISLA)University of AmsterdamAmsterdamThe Netherlands

Personalised recommendations