Advertisement

The GIRT Data in the Evaluation of CLIR Systems – from 1997 Until 2003

  • Michael Kluck
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3237)

Abstract

The motivations behind the creation of the German Indexing and Retrieval Test database (GIRT) are described and an overview of the structure of the different versions of GIRT is given. The way in which GIRT has been employed in various TREC and CLEF campaigns is then illustrated with a short description of methods and procedures used. The paper concludes with a summary of the trends in the GIRT tracks of these evaluation campaigns.

Keywords

Information Retrieval Machine Translation Information Retrieval System Parallel Corpus Bilingual Lexicon 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Womser-Hacker, C.: Das MIMOR-Modell. Mehrfachindexierung zur dynamischen Methoden-Objekt-Relationierung im Information Retrieval. Habilitationsschrift. Universität Regensburg (1996)Google Scholar
  2. 2.
    Buckley, C., Singhal, A., Mitra, M., Salton, G.: New Retrieval Approaches Using SMART. In: Harman, D. (ed.) The Fourth Text Retrieval Conference (TREC-4) [1995]. NIST, Gaithersburg (1996)Google Scholar
  3. 3.
    Haas, S.W.: Disciplinary Variation in Automatic Sublanguage Term Identification. Journal of the American Society for Information Science 48(1), 67–79 (1997)CrossRefGoogle Scholar
  4. 4.
    Kluck, M.: Die Evaluation von Cross-Language-Retrieval-Systemen mit Hilfe der GIRT-Daten des IZ. Ein Bericht über die Entwicklung im Zeitraum von 1997 bis 2003. Informationszentrum Sozialwissenschaften, Bonn (2003)Google Scholar
  5. 5.
    Frisch, E., Kluck, M.: Pretest zum Projekt German Indexing and Retrieval Testdatabase (GIRT) unter Anwendung der Retrievalsysteme Messenger und freeWAISsf, 2nd edn. Informationszentrum Sozialwissenschaften, Bonn (1997)Google Scholar
  6. 6.
    Kluck, M.: German Indexing and Retrieval Test Data Base (GIRT): Some Results of the Pre-test. In: Dunlop, M.D. (ed.) The 20th BCS IRSG Colloquium: Discovering New Worlds of IR (IRSG 1998), Grenoble, France, March 25-27 (1998) (= electronic workshops in computing), see: http://www.ewic.org.uk/ewic/workshop/view.cfm/IRSG-98
  7. 7.
    Krause, J., Mutschke, P.: Indexierung und Fulcrum-Evaluierung. Informationszentrum Sozialwissenschaften, Bonn (1999), see: http://www.gesis.org/Publikationen/Berichte/IZ_Arbeitsberichte/pdf/ab17.pdf
  8. 8.
    Binder, G., Stahl, M., Faulbaum, L.: Vergleichsuntersuchung MESSENGER – FULCRUM. Informationszentrum Sozialwissenschaften, Bonn (2000), see: http://www.gesis.org/Publikationen/Berichte/IZ_Arbeitsberichte/pdf/ab18.pdf
  9. 9.
    Womser-Hacker, C. (ed.) u.a.: Projektkurs Informationsmanagement: Durchführung einer Evaluierungsstudie, Vergleich der Information-Retrieval-Systeme (IRS) DOMESTIC - LARS II - TextExtender. Universität Konstanz (1998)Google Scholar
  10. 10.
    Käter, T., Rittberger, M., Womser-Hacker, C.: Evaluierung der Text-Retrievalsysteme Domestic, Intelligent Miner for Text, Lars II und TextExtender. In: Semar, W., Kuhlen, R. (eds.) Information Engineering. Proceedings des 4. Konstanzer Informationswissenschaftlichen Kolloquiums (KIK 1999), pp. 63–73. UVK, Konstanz (1999)Google Scholar
  11. 11.
    Käter, T.: Evaluierung des Text-Retrievalsystems Intelligent Miner for Text von IBM - Eine Studie im Vergleich zur Evaluierung anderer Systeme. Universität Konstanz (1999), http://www.ub.uni-konstanz.de/kops/volltexte/1999/283/
  12. 12.
    Griesbaum, J.: Evaluierung hybrider Suchsysteme im WWW. Diplomarbeit Informationswissenschaft Universität Konstanz (2000)Google Scholar
  13. 13.
    Eibl, M.: Visualisierung im Document Retrieval. Theoretische und praktische Zusammenführung von Softwareergonomie und Graphik Design. Informationszentrum Sozialwissenschaften, Bonn (2000)Google Scholar
  14. 14.
    Mandl, T.: Tolerantes Information Retrieval. Neuronale Netze zur Erhöhung der Adaptivität und Flexibilität bei der Informationssuche. UVK, Konstanz (2001)Google Scholar
  15. 15.
    Kluck, M., Gey, F.C.: The Domain-Specific Task of CLEF - Specific Evaluation Strategies in Cross-Language Information Retrieval. In: Peters, C. (ed.) CLEF 2000. LNCS, vol. 2069, pp. 48–56. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  16. 16.
    Schott, H. (ed.): Thesaurus Sozialwissenschaften – Thesaurus for the Social Sciences, Ausgabe edn. (1999), [Bd. 1:] Deutsch-Englisch – German-English, [Bd. 2] Englisch-Deutsch – English-German. Informationszentrum Sozialwissenschaften, Bonn (1999) Google Scholar
  17. 17.
    Schott, H. (ed.): Thesaurus Sozialwissenschaften (Ausgabe) (1999), [Bd. 1:] Alphabetischer Teil, [Bd. 2] Systematischer Teil. Informationszentrum Sozialwissenschaften, Bonn (1999) Google Scholar
  18. 18.
    Basarnova, S., Magaj, H., Mdivani, R., Schott, H., Sucker, D. (eds.): Thesaurus Sozialwissenschaften Bd.1: Deutsch-Englisch-Russisch, Bd. 2: Russisch-Deutsch-Englisch, Bd. 3: Register. Informationszentrum Sozialwissenschaften / Institut für wissenschaftliche Information in den Gesellschaftswissenschaften (INION RadW), Bonn, Moskau (1997)Google Scholar
  19. 19.
    Kando, N.: CLIR at NTCIR Workshop 3: Cross-Language and Cross-Genre Retrieval. In: Peters, C., Braschler, M., Gonzalo, J. (eds.) CLEF 2002. LNCS, vol. 2785, pp. 485–504. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  20. 20.
    Harman, D., Braschler, M., Hess, M., Kluck, M., Peters, C., Schäuble, P., Sheridan, P.: CLIR Evaluation at TREC. In: Peters, C. (ed.) CLEF 2000. LNCS, vol. 2069, pp. 7–23. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  21. 21.
    Vorhees, E.M., Harman, D.K. (eds.): The Eighth Text Retrieval Conference (TREC8) (1999). NIST, Gaithersburg (2000), at: http://trec.nist.gov/pubs/trec8/t8_proceedings.html
  22. 22.
    Cooper, W., Chen, A., Gey, F.: Full Text Retrieval based on Probabilistic Equations with Coefficients fitted by Logistic Regression. In: Harman, D.K. (ed.) The Second Text Retrieval Conference (TREC-2), pp. 57–66. NIST, Gaithersburg (1994)Google Scholar
  23. 23.
    Gey, F.C., Jiang, H.: English-German Cross-Language Retrieval for the GIRT Collection - Exploiting a Multilingual Thesaurus. In: Vorhees, E.M., Harman, D.K. (eds.) The Eighth Text Retrieval Conference (TREC8) [1999], pp. 301–306. NIST, Gaithersburg (2000)Google Scholar
  24. 24.
    Braschler, M., Schäuble, P., Kan, M.-Y., Klavans, J.L.: The Eurospider Retrieval System and the TREC-8 Cross-Language Track. In: Vorhees, E.M., Harman, D.K. (eds.) The Eighth Text Retrieval Conference (TREC8) [1999], pp. 367–376. NIST, Gaithersburg (2000)Google Scholar
  25. 25.
    Sheridan, P., Ballerini, J.P.: Experiments in Multilingual Information Retrieval Using the SPIDER System. In: Proceeding of the 19th Annual International ACM SIGIR Conference, pp. 58–65 (1996)Google Scholar
  26. 26.
    Gey, F.C., Jiang, H., Petras, V., Chen, A.: Cross-Language Retrieval for the CLEF Collections – Comparing Multiple Methods of Retrieval. In: Peters, C. (ed.) CLEF 2000. LNCS, vol. 2069, pp. 116–128. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  27. 27.
    Gövert, N.: Bilingual Information Retrieval with HyREX and Internet Translations Services. In: Peters, C. (ed.) CLEF 2000. LNCS, vol. 2069, pp. 237–244. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  28. 28.
    Gey, F.C., Jiang, H., Perelman, N.: Working with Russian Queries for the GIRT, Bilingual, and Multilingual CLEF Tasks. In: Peters, C., Braschler, M., Gonzalo, J., Kluck, M. (eds.) CLEF 2001. LNCS, vol. 2406, pp. 235–243. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  29. 29.
    Gey, F.C.: Research to Improve Cross-Language Retrieval – Position Paper for CLEF. In: Peters, C. (ed.) CLEF 2000. LNCS, vol. 2069, pp. 83–88. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  30. 30.
    Kamps, J., Monz, C., de Rijke, M.: Combining Evidence for Cross-Language Information retrieval. In: Peters, C., Braschler, M., Gonzalo, J. (eds.) CLEF 2002. LNCS, vol. 2785, pp. 111–126. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  31. 31.
    Petras, V., Perelman, N., Gey, F.C.: Using Thesauri in Cross-Language retrieval for German and French. In: Peters, C., Braschler, M., Gonzalo, J. (eds.) CLEF 2002. LNCS, vol. 2785, pp. 349–362. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  32. 32.
    Renders, J.-M., Déjean, H., Gaussier, E.: Assessing Automatically Extracted Bilingual Lexicons for CLIR in Vertical Domains: XRCE Participation in the GIRT Track of CLEF-2002. In: Peters, C., Braschler, M., Gonzalo, J. (eds.) CLEF 2002. LNCS, vol. 2785, pp. 363–371. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  33. 33.
    Hackl, R., Kölle, R., Mandl, T., Womser-Hacker, C.: Domain Specific Retrieval Experiments with MIMOR at the University of Hildesheim. In: Peters, C., Braschler, M., Gonzalo, J. (eds.) CLEF 2002. LNCS, vol. 2785, pp. 343–348. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  34. 34.
    Kamps, J., Monz, C., de Rijke, M., Sigurbjörnsson, B.: Language-Dependent and Language-Independent Approaches to Cross-Lingual Text Retrieval. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 152–165. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  35. 35.
    Petras, V., Perelman, N., Gey, F.C.: UC Berkeley at CLEF-2003 – Russian Language Experiments and Domain-Specific Retrieval. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 401–411. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  36. 36.
    Alderuccio, D., Bordoni, L., Loretto, V.: Data Compression Approach to the Monolingual GIRT Task: An Agnostic Point of View. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 391–400. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  37. 37.
    Leveling, J.: University of Hagen at CLEF 2003: Natural Language Access to the GIRT4 Data. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 412–424. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  38. 38.
    Braschler, M., Peters, C.: Cross-Language Evaluation Forum: Objectives, Results, Achievements. Information Retrieval 7, 1–2, 7–31 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Michael Kluck
    • 1
  1. 1.Informationszentrum Sozialwissenschaften (IZ)BonnGermany

Personalised recommendations