The Evaluation of Semantic Tools to Support Physicians in the Extraction of Diagnosis Codes
Over the past few years the extraction of medical information from German medical reports by means of semantic approaches and algorithms has been an increasing area of research. Currently, several tools are available that aim to support the physician in different ways. We developed a method to evaluate these tools in their ability to extract information from large amounts of data. We tested two off-the-shelf tools that worked in a background mode. We found that the field of quality management made it necessary that these large amounts of data could be background or batch processed. Additionally, we developed a metric, based on the semantic distance of the ICD codes, in order to improve the comparison of the accuracy of the codes suggested by the tools. The results of our evaluation showed that, at present, the tools are capable of supporting inexperienced physicians, however are still not sophisticated enough to work without human interaction.
KeywordsHuman Language Analysis and Natural Language Processing Evaluation Semantics
Unable to display preview. Download preview PDF.
- 3.Ruch, P., Baud, R., Geissbuhler, A.: Learning-free text categorization. In: Artificial Intelligence in Medicine, Proceedings, pp. 199–208. Springer, Berlin (2003)Google Scholar
- 4.Geierhofer, R., Holzinger, A.: Creating an Annotated Set of Medical Reports to Evaluate Information Retrieval Techniques. In: SEMANTICS 2007, Graz, Austria, September 5-7, 2007, pp. 331–339 (2007)Google Scholar
- 5.Holzinger, A., Geierhofer, R., Errath, M.: Semantic Information in Medical Information Systems - from Data and Information to Knowledge: Facing Information Overload. In: Proceedings of I-MEDIA 2007 and I-SEMANTICS 2007, pp. 323–330 (2007)Google Scholar
- 6.Matykiewicz, P., Duch, W., Pestian, J.: Nonambiguous concept mapping in medical domain, In: Artificial Intelligence and Soft Computing. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2006. LNCS (LNAI), vol. 4029, pp. 941–950. Springer, Heidelberg (2006)CrossRefGoogle Scholar
- 7.Schulz, S., Hanser, S., Hahn, U., Rogers, J.: The semantics of procedures and diseases in SNOMED (R) CT. Methods of Information in Medicine 45(4), 354–358 (2006)Google Scholar