Abstract
This paper focuses on the proposal and evaluation of two multimodal fusion techniques in the field of Visual Information Retrieval (VIR). These proposals are based on two widely used fusion strategies in the VIR area, the multimodal blind relevance feedback and the multimodal re-ranking strategy. Unlike the existent techniques, our alternative proposals are guided by the evidence found in the natural language annotations related to the images. The results achieved by our runs in two different ImageCLEF tasks, 3rd place in the Wikipedia task [1] and 4th place within all the automatic runs in the photo task [2], jointly with the results obtained in later experiments presented in this paper show us that the use of conceptual information associated with an image can improve significantly the performance of the original multimodal fusion techniques used.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Navarro, S., Muñoz, R., Llopis, F.: A Textual Approach based on Passages Using IR-n in WikipediaMM Task 2008. In: Online Working Notes, CLEF 2008 (2008)
Navarro, S., Llopis, F., Muñoz, R.: Different Multimodal Approaches using IR-n in ImageCLEFphoto 2008. In: Online Working Notes, CLEF 2008 (2008)
Grubinger, M.: Analysis and Evaluation of Visual Information Systems Performance. PhD thesis, Engineering and Science Victoria University (2007)
Díaz-Galiano, M., García-Cumbreras, M., Martín-Valdivia, M., Montejo-Raez, A., Urea-López, L.: Sinai at imageclef 2007. In: Working Notes of the 2007 CLEF Workshop, Budapest, Hungary (September 2007)
Gao, S., Chevallet, J.P., Le, T.H.D., Pham, T.T., Lim, J.H.: Ipal at imageclef 2007 mixing features, models and knowledge. In: Working Notes of the 2007 CLEF Workshop, Budapest, Hungary (September 2007)
Robertson, S.E., Sparck Jones, K.: Relevance weighting of search terms. Journal of the American Society for Information Science 27(3), 129–146 (1977)
Xu, J., Croft, W.B.: Improving the effectiveness of information retrieval with local context analysis. ACM Trans. Inf. Syst. 18(1), 79–112 (2000)
Llopis, F., Vicedo, J.L., Ferrández, A.: IR-n System at CLEF-2002. In: Peters, C., Braschler, M., Gonzalo, J. (eds.) CLEF 2002. LNCS, vol. 2785, pp. 291–300. Springer, Heidelberg (2003)
Navarro, S., Muñoz, R., Llopis, F.: A Multimodal Approach to the Medical Retrieval Task using IR-n. In: Online Working Notes, CLEF 2008 (2008)
Liu, H., Junzhong Gu, Z.L.: Improving the Effectiveness of Local Context Analysis Based on Semantic Similarity. In: 2007 International Conference on Convergence Information Technology, ICCIT 2007 (2007)
Reid, N.: The photographic collections in st andrews university library. Scottish Archives 5, 83–90 (1999)
Clough, P., Grubinger, M., Deselaers, T., Hanbury, A., Müller, H.: Overview of the imageclef 2006 photographic retrieval and object annotation tasks. In: Peters, C., Clough, P., Gey, F.C., Karlgren, J., Magnini, B., Oard, D.W., de Rijke, M., Stempfhuber, M. (eds.) CLEF 2006. LNCS, vol. 4730, pp. 579–594. Springer, Heidelberg (2007)
Grubinger, M., Clough, P., Hanbury, A., Müller, H.: Overview of the ImageCLEFphoto 2007 photographic retrieval task. In: Peters, C., Jijkoun, V., Mandl, T., Müller, H., Oard, D.W., Peñas, A., Petras, V., Santos, D. (eds.) CLEF 2007. LNCS, vol. 5152, pp. 433–444. Springer, Heidelberg (2008)
Tsikrika, T., Kludas, J.: Overview of the wikipediaMM task at ImageCLEF 2008. In: Peters, C., Giampiccol, D., Ferro, N., Petras, V., Gonzalo, J., Peñas, A., Deselaers, T., Mandl, T., Jones, G., Kurimo, M. (eds.) Evaluating Systems for Multilingual and Multimodal Information Access. LNCS, vol. 5706, pp. 539–550. Springer, Heidelberg (2009)
Amati, G., Van Rijsbergen, C.J.: Probabilistic Models of information retrieval based on measuring the divergence from randomness. ACM TOIS 20(4), 357–389 (2002)
Deselaers, T., Keysers, D., Ney, H.: Features for image retrieval: An experimental comparison. Information Retrieval 11(2), 77–107 (2008)
Demerdash, O.E., Kosseim, L., Bergler, S.: CLaC at ImageCLEFPhoto 2008. In: Online Working Notes, CLEF 2008 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Navarro, S., Llopis, F., Muñoz, R. (2010). Using Evidences Based on Natural Language to Drive the Process of Fusing Multimodal Sources. In: Horacek, H., Métais, E., Muñoz, R., Wolska, M. (eds) Natural Language Processing and Information Systems. NLDB 2009. Lecture Notes in Computer Science, vol 5723. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12550-8_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-12550-8_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12549-2
Online ISBN: 978-3-642-12550-8
eBook Packages: Computer ScienceComputer Science (R0)