The Evaluation of Semantic Tools to Support Physicians in the Extraction of Diagnosis Codes

  • Regina Geierhofer
  • Andreas Holzinger
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4799)


Over the past few years the extraction of medical information from German medical reports by means of semantic approaches and algorithms has been an increasing area of research. Currently, several tools are available that aim to support the physician in different ways. We developed a method to evaluate these tools in their ability to extract information from large amounts of data. We tested two off-the-shelf tools that worked in a background mode. We found that the field of quality management made it necessary that these large amounts of data could be background or batch processed. Additionally, we developed a metric, based on the semantic distance of the ICD codes, in order to improve the comparison of the accuracy of the codes suggested by the tools. The results of our evaluation showed that, at present, the tools are capable of supporting inexperienced physicians, however are still not sophisticated enough to work without human interaction.


Human Language Analysis and Natural Language Processing Evaluation Semantics 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Stausberg, J., Koch, D., Ingenerf, J., Betzler, M.: Comparing paper-based with electronic patient records: Lessons learned during a study on diagnosis and procedure codes. Journal of the American Medical Informatics Association 10(5), 470–477 (2003)CrossRefGoogle Scholar
  2. 2.
    Holzinger, A., Geierhofer, R., Errath, M.: Semantische Informationsextraktion in medizinischen Informationssystemen. Informatik Spektrum 30(2), 69–78 (2007)CrossRefGoogle Scholar
  3. 3.
    Ruch, P., Baud, R., Geissbuhler, A.: Learning-free text categorization. In: Artificial Intelligence in Medicine, Proceedings, pp. 199–208. Springer, Berlin (2003)Google Scholar
  4. 4.
    Geierhofer, R., Holzinger, A.: Creating an Annotated Set of Medical Reports to Evaluate Information Retrieval Techniques. In: SEMANTICS 2007, Graz, Austria, September 5-7, 2007, pp. 331–339 (2007)Google Scholar
  5. 5.
    Holzinger, A., Geierhofer, R., Errath, M.: Semantic Information in Medical Information Systems - from Data and Information to Knowledge: Facing Information Overload. In: Proceedings of I-MEDIA 2007 and I-SEMANTICS 2007, pp. 323–330 (2007)Google Scholar
  6. 6.
    Matykiewicz, P., Duch, W., Pestian, J.: Nonambiguous concept mapping in medical domain, In: Artificial Intelligence and Soft Computing. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2006. LNCS (LNAI), vol. 4029, pp. 941–950. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  7. 7.
    Schulz, S., Hanser, S., Hahn, U., Rogers, J.: The semantics of procedures and diseases in SNOMED (R) CT. Methods of Information in Medicine 45(4), 354–358 (2006)Google Scholar
  8. 8.
    Senvar, M., Bener, A.: Matchmaking of semantic web services using semantic-distance information. In: Yakhno, T., Neuhold, E.J. (eds.) ADVIS 2006. LNCS, vol. 4243, pp. 177–186. Springer, Heidelberg (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Regina Geierhofer
    • 1
  • Andreas Holzinger
    • 1
  1. 1.Institute for Medical Informatics, Statistics & Documentation (IMI), Research Unit HCI4MED, Medical University Graz, A-8036 GrazAustria

Personalised recommendations