Automatic Eye-Tracking-Assisted Chest Radiography Pathology Screening

Abstract

Chest radiography is increasingly used worldwide to diagnose a series of illnesses targeting the lungs and heart. The high amount of examinations leads to a severe burden on radiologists, which benefit from the introduction of artificial intelligence tools in clinical practice, such as deep learning classification models. Nevertheless, these models are undergoing limited implementation due to the lack of trustworthy explanations that provide insights about their reasoning. In an attempt to increase the level of explainability, the deep learning approaches developed in this work incorporate in their decision process eye-tracking data collected from experts. More specifically, eye-tracking data is used in the form of heatmaps to change the input to the selected classifier, an EfficientNet-b0, and to guide its focus towards relevant parts of the images. Prior to the classification task, UNet-based models are used to perform heatmap reconstruction, making this framework independent of eye-tracking data during inference. The two proposed approaches are applied to all existing public eye-tracking datasets, to our knowledge, regarding chest X-ray screening, namely EGD, REFLACX and CXR-P. For these datasets, the reconstructed heatmaps highlight important anatomical/pathological regions and the area under the curve results are comparable to the state-of-the-art and to the considered baseline. Furthermore, the quality of the explanations derived from the classifier is superior for one of the approaches, which can be attributed to the use of eye-tracking data.

This work was funded by the ERDF - European Regional Development Fund, through the Programa Operacional Regional do Norte (NORTE 2020) and by National Funds through the FCT - Portuguese Foundation for Science and Technology, I.P. within the scope of the CMU Portugal Program (NORTE-01-0247-FEDER-045905) and LA/P/0063/2020.