Comparative Evaluation of Multilingual Information Access Systems

4th Workshop of the Cross-Language Evaluation Forum, CLEF 2003, Trondheim, Norway, August 21-22, 2003, Revised Selected Papers

  • Carol Peters
  • Julio Gonzalo
  • Martin Braschler
  • Michael Kluck
Conference proceedings CLEF 2003

Part of the Lecture Notes in Computer Science book series (LNCS, volume 3237)

Table of contents

  1. Front Matter
  2. Introduction

    1. Julio Gonzalo, Carol Peters
      Pages 1-6
    2. Martin Braschler, Carol Peters
      Pages 7-20
  3. Part I. Ad-hoc Text Retrieval Tracks

    1. Martin Braschler
      Pages 44-63
  4. Mainly Cross-Language Experiments

    1. Jacques Savoy
      Pages 64-73
    2. Eija Airio, Heikki Keskustalo, Turid Hedlund, Ari Pirkola
      Pages 74-84
    3. Paul McNamee, James Mayfield
      Pages 85-97
    4. Nicola Cancedda, Hervé Déjean, Éric Gaussier, Jean-Michel Renders, Alexei Vinokourov
      Pages 98-107
    5. Fernando Llopis, Rafael Muñoz
      Pages 122-132
    6. Jaap Kamps, Christof Monz, Maarten de Rijke, Börkur Sigurbjörnsson
      Pages 152-165
    7. René Hackl, Ralph Kölle, Thomas Mandl, Alexandra Ploedt, Jan-Hendrik Scheufen, Christa Womser-Hacker
      Pages 166-173
    8. Romaric Besançon, Gaël de Chalendar, Olivier Ferret, Christian Fluhr, Olivier Mesnard, Hubert Naets
      Pages 174-184
    9. Fernando Martínez-Santiago, Arturo Montejo-Ráez, Luis Alfonso Ureña-López, M. Carlos Díaz-Galiano
      Pages 192-201
    10. Wen-Cheng Lin, Hsin-Hsi Chen
      Pages 202-209
    11. José L. Martínez, Julio Villena, Jorge Fombella, Ana G. Serrano, Paloma Martínez, Josê M. Goñi et al.
      Pages 210-219
    12. Giorgio M. Di Nunzio, Nicola Ferro, Massimo Melucci, Nicola Orio
      Pages 220-235
    13. Yan Qu, Gregory Grefenstette, David A. Evans
      Pages 236-241
    14. Gilles Sérasset, Jean-Pierre Chevallet
      Pages 242-252
    15. Michael P. Oakes, Souvik Banerjee
      Pages 263-270
  5. Monolingual Experiments

    1. Roel Brand, Marvin Brünner, Samuel Driessen, Pascha Iljin, Jakob Klok
      Pages 301-309
    2. Gianni Amati, Claudio Carpineto, Giovanni Romano
      Pages 310-318
    3. Rickard Cöster, Magnus Sahlgren, Jussi Karlgren
      Pages 337-344
    4. Jesús Vilares, Miguel A. Alonso, Francisco J. Ribadas
      Pages 345-357
    5. Javier Fernández, Ricardo Mones, Irene Díaz, José Ranilla, Elías F. Combarro
      Pages 358-366
    6. Yuichi Kojima, Hideo Itoh, Hiroko Mano, Yasushi Ogawa
      Pages 367-372
    7. Peter van der Weerd
      Pages 373-375
  6. Part II. Domain-Specific Document Retrieval

  7. Part III. Interactive Cross-Language Retrieval

    1. Douglas W. Oard, Julio Gonzalo
      Pages 425-434
    2. Bonnie Dorr, Daqing He, Jun Luo, Douglas W. Oard, Richard Schwartz, Jianqiang Wang et al.
      Pages 435-449
    3. Fernando López-Ostenero, Julio Gonzalo, Felisa Verdejo
      Pages 450-461
    4. Jussi Karlgren, Preben Hansen
      Pages 468-470
  8. Part IV. Cross-Language Question Answering

    1. Bernardo Magnini, Simone Romagnoli, Alessandro Vallin, Jesús Herrera, Anselmo Peñas, Víctor Peinado et al.
      Pages 471-486
    2. Bernardo Magnini, Simone Romagnoli, Alessandro Vallin, Jesús Herrera, Anselmo Peñas, Víctor Peinado et al.
      Pages 487-500
    3. Matteo Negri, Hristo Tanev, Bernardo Magnini
      Pages 501-513
    4. Abdessamad Echihabi, Douglas W. Oard, Daniel Marcu, Ulf Hermjakob
      Pages 514-522

About these proceedings


The fourth campaign of the Cross-language Evaluation Forum (CLEF) for European languages was held from January to August 2003. Participation in this campaign showed a slight rise in the number of participants from the previous year, with 42 groups submitting results for one or more of the different tracks (compared with 37 in 2002), but a steep rise in the number of experiments attempted. A distinctive feature of CLEF 2003 was the number of new tracks and tasks that were offered as pilot experiments. The aim was to try out new ideas and to encourage the development of new evaluation methodologies, suited to the emerging requirements of both system developers and users with respect to today’s digital collections and to encourage work on many European languages rather than just those most widely used. CLEF is thus gradually pushing its participants towards the ultimate goal: the development of truly multilingual systems capable of processing collections in diverse media. The campaign culminated in a two-day workshop held in Trondheim, Norway, 21–22 August, immediately following the 7th European Conference on Digital Libraries (ECDL 2003), and attended by more than 70 researchers and system developers. The objective of the workshop was to bring together the groups that had participated in the CLEF 2003 campaign so that they could report on the results of their experiments.


DOM algorithms bilingual information retrieval cross-language queries cross-lingual text retrieval image retrieval information system interactive text retrieval systems machine translation multilingual information retrieval natural language processing ontology proving spoken document retrieval

Editors and affiliations

  • Carol Peters
    • 1
  • Julio Gonzalo
    • 2
  • Martin Braschler
    • 2
  • Michael Kluck
    • 3
  1. 1.ISTI-CNR, Area di RicercaPisaItaly
  2. 2.No Affiliations 
  3. 3.German Institute for International and Security AffairsStiftung Wissenschaft und Politik (SWP)BerlinGermany

Bibliographic information

  • DOI
  • Copyright Information Springer-Verlag Berlin Heidelberg 2004
  • Publisher Name Springer, Berlin, Heidelberg
  • eBook Packages Springer Book Archive
  • Print ISBN 978-3-540-24017-4
  • Online ISBN 978-3-540-30222-3
  • Series Print ISSN 0302-9743
  • Series Online ISSN 1611-3349
  • Buy this book on publisher's site