Attention-Guided Curriculum Learning for Weakly Supervised Classification and Localization of Thoracic Diseases on Chest Radiographs
In this work, we exploit the task of joint classification and weakly supervised localization of thoracic diseases from chest radiographs, with only image-level disease labels coupled with disease severity-level (DSL) information of a subset. A convolutional neural network (CNN) based attention-guided curriculum learning (AGCL) framework is presented, which leverages the severity-level attributes mined from radiology reports. Images in order of difficulty (grouped by different severity-levels) are fed to CNN to boost the learning gradually. In addition, highly confident samples (measured by classification probabilities) and their corresponding class-conditional heatmaps (generated by the CNN) are extracted and further fed into the AGCL framework to guide the learning of more distinctive convolutional features in the next iteration. A two-path network architecture is designed to regress the heatmaps from selected seed samples in addition to the original classification task. The joint learning scheme can improve the classification and localization performance along with more seed samples for the next iteration. We demonstrate the effectiveness of this iterative refinement framework via extensive experimental evaluations on the publicly available ChestXray14 dataset. AGCL achieves over 5.7% (averaged over 14 diseases) increase in classification AUC and 7%/11% increases in Recall/Precision for the localization task compared to the state of the art.
This research was supported by the Intramural Research Program of the National Institutes of Health Clinical Center and by the Ping An Technology Co., Ltd. through a Cooperative Research and Development Agreement. The authors thank NVIDIA for GPU donation.
- 1.Bengio, Y., Louradour, J., et al.: Curriculum learning. In: ICML (2009)Google Scholar
- 2.He, K., et al.: Deep residual learning for image recognition. In: IEEE CVPR (2016)Google Scholar
- 3.Jin, D., Xu, Z., et al.: CT-realistic lung nodule simulation from 3D conditional generative adversarial networks for robust lung segmentation. In: MICCAI (2018)Google Scholar
- 4.Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS (2012)Google Scholar
- 6.Li, Z., Wang, C., Han, M., Xue, Y., Wei, W., Li, L.J., Fei-Fei, L.: Thoracic disease identification and localization with limited supervision. In: IEEE CVPR (2018)Google Scholar
- 7.Rajpurkar, P., Irvin, J., Zhu, K., Yang, B., et al.: CheXNet: radiologist-level pneumonia detection on chest X-rays with deep learning. arXiv:1711.05225 (2017)
- 8.Shi, M., Ferrari, V.: Weakly supervised object localization using size estimates. In: ECCV (2016)Google Scholar
- 9.Tang, Y., Wang, J., Gao, B., et al.: Large scale semi-supervised object detection using visual and semantic knowledge transfer. In: IEEE CVPR (2016)Google Scholar
- 10.Tang, Y., et al.: Semi-automatic RECIST labeling on CT scans with cascaded convolutional neural networks. In: MICCAI (2018)Google Scholar
- 11.Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: ChestX-ray8: hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: IEEE CVPR (2017)Google Scholar
- 12.Wang, X., Peng, Y., et al.: TieNet: text-image embedding network for common thorax disease classification and reporting in chest X-rays. In: IEEE CVPR (2018)Google Scholar
- 14.Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: IEEE CVPR (2016)Google Scholar