Learning from Noisy Medical Data: A Comparative Study Based on a Real Diagnostic Problem

Abstract

Clinicians routinely collect extensive case histories on their patients and in certain medical domains this data may be supplemented with confirmed or “working” diagnosis obtained by patient follow-up. The possibility of using such datasets as a source of ‘knowledge’ for diagnostic systems has been a goal of many research studies.

This paper reports on the application of five approaches: rule induction, neural networks, statistically based diagnostic trees, Bayes discriminants and logistic models; to the construction of diagnostic aids based on a noisy, mainly categorical, medical dataset giving the clinical presentation of patients with either Multiple Sclerosis or Cerebrovascular(Vascular) Disease who have been referred for Magnetic Resonance Imaging.

The procedures investigated gave very similar results in terms of overall diagnostic performance although the ‘format’ of the resulting diagnostic aids was very different. The use of Multiple Correspondence Analysis as a preparatory technique, to remove noisy variables, proved very successful in identifying a smaller subset of items that were more amenable to ‘automated’ techniques such as neural networks/rule induction and also assisted in the selection of variables for statistical discrimination.