Abstract
Accuracy of current automatic image labeling methods is under the requirements of annotation-based image retrieval systems. The performance of most of these labeling methods is poor if we just consider the most relevant label for a given region. However, if we look within the set of the top− k candidate labels for a given region, accuracy of most of these systems is improved. In this paper we take advantage of this fact and propose a method (NBI) based on word co-occurrences that uses the naïve Bayes formulation for improving automatic image annotation methods. Our approach utilizes co-occurrence information of the candidate labels for a region with those candidate labels for the other surrounding regions, within the same image, for selecting the correct label. Co-occurrence information is obtained from an external collection of manually annotated images: the IAPR-TC12 benchmark. Experimental results using a k −nearest neighbors method as our annotation system, give evidence of significant improvements after applying the NBI method. NBI is efficient since the co-occurrence information was obtained off-line. Furthermore, our method can be applied to any other annotation system that ranks labels by their relevance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barnard, K., Forsyth, D.: Learning the semantics of words and pictures. In: Proc. ICCV, vol. 2, pp. 408–415. IEEE, Los Alamitos (2001)
Blei, D.M., Jordan, M.I.: Modeling annotated data. In: Proc. of the 26th international ACM-SIGIR conf. on Research and development in informaion retrieval, pp. 127–134. ACM Press, New York, NY, USA (2003)
Carbonetto, P.: Unsupervised statistical models for general object recognition. Master’s thesis, C.S. Department, University of British Columbia (August 2003)
Carbonetto, P., de Freitas, N., Barnard, K.: A statistical model for general context object recognition. In: Proc. of 8th ECCV, pp. 350–362 (2005)
Carbonetto, P., de Freitas, N., Gustafson, P., Thompson, N.: Bayesian feature eeighting for unsupervised learning. In: Proc. of the HLT-NAACL workshop on Learning word meaning from non-linguistic data, Morristown, NJ, USA, pp. 54–61 (2003)
Carneiro, G., Chan, A.B., Moreno, P.J., Vasconcelos, N.: Supervised learning of semantic classes for image annotation and retrieval. IEEE Trans. on PAMI 29(3), 394–410 (2007)
Carneiro, G., Vasconcelos, N.: Formulating semantic image annotation as a supervised learning problem. In: Proc. of CVPR, Washington, DC, USA, vol. 2, pp. 163–168. IEEE Computer Society, Los Alamitos (2005)
Chen, S.F., Goodman, J.: An empirical study of smoothing techniques for language modeling. In: Proc. of the 34th meeting on Association for Computational Linguistics, Morristown, NJ, USA, pp. 310–318 (1996)
Datta, R., Li, J., Wang, J.Z.: Content-based image retrieval - approaches and trends of the new age. In: Proceedings ACM International Workshop on Multimedia Information Retrieval, Singapore. ACM Multimedia, New York (2005)
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley-Interscience, Chichester (2000)
Duygulu, P., Barnard, K., de Freitas, N., Forsyth, D.: Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2353, pp. 97–112. Springer, Heidelberg (2002)
Iyengar, G., et al.: Joint visual-text modeling for automatic retrieval of multimedia documents. In: Proc. the 13th MULTIMEDIA, pp. 21–30. ACM Press, New York, NY, USA (2005)
Ghoshal, A., Ircing, P., Khudanpur, S.: Hmm’s for automatic annotation and content-based retrieval of images and video. In: Proc. of the 28th int. conf. on Research and development in information retrieval, New York, NY, USA, pp. 544–551 (2005)
Grubinger, M., Clough, P., Leung, C.: The iapr tc-12 benchmark -a new evaluation resource for visual information systems. In: Proc. of the International Workshop OntoImage 2006 Language Resources for CBIR (2006)
Hare, J.S., Lewis, P.H., Enser, P.G.B., Sandom, C.J.: Mind the Gap: Another look at the problem of the semantic gap in image retrieval. In: Hanjalic, A., Chang, E.Y., Sebe, N. (eds.) Proceedings of Multimedia Content Analysis, Management and Retrieval, San Jose, California, USA, SPIE, vol. 6073 (2006)
Lavrenko, V., Manmatha, R., Jeon, J.: A model for learning the semantics of pictures. In: NIPS, vol. 16. MIT Press, Cambridge, MA (2004)
Li, W., Sun, M.: Automatic image annotation based on wordnet and hierarchical ensembles. In: Gelbukh, A. (ed.) CICLING 2006. LNCS, vol. 3878, pp. 417–428. Springer, Heidelberg (2006)
Liu, Y., Zhang, D., Lu, G., Ma, W.Y.: A survey of content-based image retrieval with high-level semantics. Pattern Recogn. 40(1), 262–282 (2007)
Mitchell, T.: Machine Learning. McGraw-Hill Education, New York (1997)
Mori, Y., Takahashi, H., Oka, R.: Image-to-word transformation based on dividing and vector quantizing images with words. In: 1st Int. Worksh. on Multimedia Intelligent Storage and Retrieval Management (1999)
Nigam, K., McCallum, A., Thrun, S., Mitchell, T.: Text classification from labeled and unlabeled documents using em. Machine Learning 39, 103–134 (2000)
Pan, J., Yang, H., Duygulu, P., Faloutsos, C.: Automatic image captioning. In: Proc. of the ICME (2004)
Shi, J., Malik, J.: Normalized cuts and image segmentation. PAMI-IEEE 22(8), 888–905 (2000)
Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Trans. on PAMI 22(12), 1349–1380 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Escalante, H.J., Montes, M., Sucar, L.E. (2008). Improving Automatic Image Annotation Based on Word Co-occurrence. In: Boujemaa, N., Detyniecki, M., Nürnberger, A. (eds) Adaptive Multimedia Retrieval: Retrieval, User, and Semantics. AMR 2007. Lecture Notes in Computer Science, vol 4918. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-79860-6_5
Download citation
DOI: https://doi.org/10.1007/978-3-540-79860-6_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-79859-0
Online ISBN: 978-3-540-79860-6
eBook Packages: Computer ScienceComputer Science (R0)