Abstract
The increasing demand on both practitioners and librarians to encode medical documents with controlled vocabularies calls for automatic tools and methods to help them perform this task efficiently. This paper presents the Benchmark evaluation of the French MeSH indexing systems carried out under the umbrella of the VUMeF consortium. The CISMeF, NOMINDEX and HONMeSHMapper systems are introduced, and evaluated on a set of 82 resources randomly taken from the CISMeF catalogue. The automatic MeSH indexing produced by each system was compared to the manual gold standard provided by the CISMeF medical librarian team. The automatic systems achieve at best a precision close to 50% at rank 1 (HONMeSHMapper, CISMeF) and HONMeSHMapper achieves the best overall F-measure. A qualitative evaluation of the indexing provided indicates that all systems tend to misevaluate the specificity of the terms to retrieve.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Humphrey, S.M., Miller, N.E.: Knowledge-Based Indexing of the Medical Literature: The Indexing Aid Project. J Am Soc Inf Sci 38(3), 184–196 (1987)
Darmoni, S.J., Consortium VUMeF: VUMeF: Extending the French Involvement in the UMLS Metathesaurus. In: AMIA Annu Symp Proc. 2003, p. 824 (2003)
Néveol, A., Rogozan, A., Darmoni, S.J.: Automatic Indexing of Online Health Resources for a French Quality Controlled Gateway. In: IP & M (2005) (in press)
Pouliquen, B., Delamarre, D., Le Beux, P.: Indexation de Textes Médicaux par Extraction de Concepts et ses Utilisations. In: JADT 2002, St Malo, France, March 2002, vol. (2), pp. 617–628 (2002)
Lenoir, P., Michel, J.R., Frangeul, C., Chales, G.: Réalisation, Développement et Maintenance de la Base de Données A.D.M. Médecine informatique 6, 51–56 (1981)
Gaudinat, A., Joubert, M., Aymard, S., Falco, L., Boyer, C., Fieschi, M.: WRAPIN: New Generation Health Search Engine Using UMLS Knowledge Sources for MeSH Term Extraction from Health Documentation. In: Medinfo. 2004, pp. 356–360 (2004)
Gaudinat, A., Boyer, C.: Automatic Extraction of MeSH Terms from MEDLINEs Abstracts. In: Workshop on Natural Language Processing in Biomedical Applications, pp. 53–57 (2002)
Cooper, G., Miller, R.: An Experiment Comparing Lexical and Statistical Methods for Extracting MeSH Terms from Clinical Free Text. J. Am. Med. Inf. Assoc. 5, 62–75 (1998)
Aronson, A.R., Mork, J.G., Gay, C.W., Humphrey, S.M., Rogers, W.J.: The NLM Indexing Initiative’s Medical Text Indexer. In: Medinfo. 2004, pp. 268–272 (2004)
Funk, M.E., Reid, C.A., Mc Googan, L.S.: Indexing Consistency in MEDLINE. Bull. Med. Libr. Assoc. 71(2), 176–183 (1983)
Mary, V., Pouliquen, B., Le Duff, F., Darmoni, S.J., Segui, A., Le Beux, P.: Automatic Conceptual Indexing of French Pharmaceutical Theses. Stud Health Technol Inform. 90, 388–392 (2002)
Névéol, A., Rogozan, A., Darmoni, S.J.: Indexation Automatique de Ressources de Santé à l’Aide de Paires de Descripteurs MeSH. In: TALN 2005 (2005) (in press)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Névéol, A., Mary, V., Gaudinat, A., Boyer, C., Rogozan, A., Darmoni, S.J. (2005). A Benchmark Evaluation of the French MeSH Indexers. In: Miksch, S., Hunter, J., Keravnou, E.T. (eds) Artificial Intelligence in Medicine. AIME 2005. Lecture Notes in Computer Science(), vol 3581. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11527770_37
Download citation
DOI: https://doi.org/10.1007/11527770_37
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-27831-3
Online ISBN: 978-3-540-31884-2
eBook Packages: Computer ScienceComputer Science (R0)