Journal of Digital Imaging

, Volume 32, Issue 6, pp 995–1007 | Cite as

Computer-Aided Diagnosis (CAD) of Pulmonary Nodule of Thoracic CT Image Using Transfer Learning

  • Shikun Zhang
  • Fengrong SunEmail author
  • Naishun Wang
  • Cuicui Zhang
  • Qianlei Yu
  • Mingqiang Zhang
  • Paul Babyn
  • Hai Zhong


Computer-aided diagnosis (CAD) has already been widely used in medical image processing. We recently make another trial to implement convolutional neural network (CNN) on the classification of pulmonary nodules of thoracic CT images. The biggest challenge in medical image classification with the help of CNN is the difficulty of acquiring enough samples, and overfitting is a common problem when there are not enough images for training. Transfer learning has been verified as reasonable in dealing with such problems with an acceptable loss value. We use the classic LeNet-5 model to classify pulmonary nodules of thoracic CT images, including benign and malignant pulmonary nodules, and different malignancies of the malignant nodules. The CT images are obtained from Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) where both pulmonary nodule scanning and nodule annotations are available. These images are labeled and stored in a medical images knowledge base (KB), which is designed and implemented in our previous work. We implement the 10-folder cross validation (CV) to testify the robustness of the classification model we trained. The result demonstrates that the transfer learning of the LeNet-5 is good for classifying pulmonary nodules of thoracic CT images, and the average values of Top-1 accuracy are 97.041% and 96.685% respectively. We believe that our work is beneficial and has potential for practical diagnosis of lung nodules.


Pulmonary nodule Classification Thoracic CT Transfer learning CNN 


Funding Information

This work is supported by the Nature Science Foundation of Shandong Province under the grant ZR2014FM006, the National Nature Science Foundation of China under the grant 81671703, and the Focus on Research and Development Plan in Shandong Province under the grant 2015GSF118026.


  1. 1.
    Hubel DH, Wiesel TN: Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J Physiol 160 (1): 106–154, 1962CrossRefGoogle Scholar
  2. 2.
    Deng J, Dong W, Socher R, Li JL, Li K, Li FF: Imagenet: a large-scale hierarchical image database. In: IEEE Conference on computer vision and pattern recognition, 2009, pp 248–255Google Scholar
  3. 3.
    Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein MS, Berg AC, Li F-F: Imagenet large scale visual recognition challenge. International Journal of Computer Vision . arXiv:, 2014
  4. 4.
    Krizhevsky A, Sutskever I, Hinton GE: Imagenet classification with deep convolutional neural networks.. In: Advances in neural information processing systems, 2012, pp 1097–1105Google Scholar
  5. 5.
    Krizhevsky A, Hinton G: Learning multiple layers of features from tiny images, 2009Google Scholar
  6. 6.
    Tran PV A fully convolutional neural network for cardiac segmentation in short-axis mri, arXiv:, 2016
  7. 7.
    Ng H-W, Nguyen VD, Vonikakis V, Winkler S: Deep learning for emotion recognition on small datasets using transfer learning. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. ACM, 2015, pp 443–449Google Scholar
  8. 8.
    Raina R, Battle A, Lee H, Packer B, Ng AY: Self-taught learning: transfer learning from unlabeled data. In: Proceedings of the 24th International Conference on Machine Learning. ACM, 2007, pp 759–766Google Scholar
  9. 9.
    Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T Decaf: A deep convolutional activation feature for generic visual recognition. In: International Conference on Machine Learning, 2014, pp 647–655Google Scholar
  10. 10.
    LeCun Y, Bottou L, Bengio Y, Haffner P: Gradient-based learning applied to document recognition. Proc IEEE 86 (11): 2278–2324, 1998CrossRefGoogle Scholar
  11. 11.
    Hu W, Huang Y, Li W, Zhang F, Li H: Deep convolutional neural networks for hyperspectral image classification. J Sensors, 2015Google Scholar
  12. 12.
    Sarraf S, Tofighi G: Deep learning-based pipeline to recognize alzheimer’s disease using fmri data. In: Future Technologies Conference, 2017Google Scholar
  13. 13.
    Armato SG III, McLennan G, Bidaut L, McNitt-Gray MF, Meyer CR, Reeves AP, ..., Kazerooni EA: The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Med Phys 38 (2): 915–931, 2011CrossRefGoogle Scholar
  14. 14.
    Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick RB, Guadarrama S, Darrell T: Caffe: convolutional architecture for fast feature embedding. ACM Multimed, arXiv:, 2014, 675–678
  15. 15.
    Shen W, Mu Z, Yang F, Yang C, Tian J: Multi-scale convolutional neural networks for lung nodule classification. In: International Conference on Information Processing in Medical Imaging. Springer, 2015, pp 588–599Google Scholar
  16. 16.
    Song Q, Zhao L, Luo X, Dou X: Using deep learning for classification of lung nodules on computed tomography images. Journal of Healthcare Engineering, 2017Google Scholar
  17. 17.
    Krewer H, Geiger B, Hall LO, Goldgof DB, Gu Y, Tockman M, Gillies RJ: Effect of texture features in computer aided diagnosis of pulmonary nodules in low-dose computed tomography. In: 2013 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2013, pp 3887–3891Google Scholar
  18. 18.
    Golan R, Jacob C, Denzinger J: Lung nodule detection in ct images using deep convolutional neural networks. In: 2016 International Joint Conference on Neural Networks (IJCNN). IEEE, 2016, pp 243–250Google Scholar
  19. 19.
    Bergtholdt M, Wiemker R, Klinder T: Pulmonary nodule detection using a cascaded svm classifier. In: Medical Imaging 2016: Computer-Aided Diagnosis. International Society for Optics and Photonics, 2016, vol 9785, pp 978513Google Scholar
  20. 20.
    Zhang C, Sun F, Zhang M, Liu W, Yu Q, Babyn P, Zhong H: Design and implementation of a medical image knowledge base for pulmonary nodules diagnosis. In: IEEE International Conference on Computer and Communications, 2018, pp 2071– 2075Google Scholar
  21. 21.
    Li Q, Cai W, Wang X, Zhou Y, Feng DD, Chen M: Medical image classification with convolutional neural network, 2014, pp 844–848Google Scholar
  22. 22.
    Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM: Deep convolutional neural networks for computer-aided detection Cnn architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35 (5): 1285, 2016CrossRefGoogle Scholar
  23. 23.
    Kuncheva LI, Juan J: . Rodriguez Classifier ensembles with a random linear oracle 19: 500–508, 2007Google Scholar
  24. 24.
    Baralis E, Chiusano S, Garza P: A lazy approach to associative classification. IEEE Trans Knowl Data Eng 20 (2): 156–171, 2008CrossRefGoogle Scholar
  25. 25.
    Zhou X: Semi - supervised learning literature survey, 2005Google Scholar
  26. 26.
    Nigam K, McCallum AK, Thrun S, Mitchell T: Text classification from labeled and unlabeled documents using em. Mach Learn 39: 103–134, 2000CrossRefGoogle Scholar
  27. 27.
    Blum A, Mitchell T: Combining labeled and unlabeled data with co-training, 1998, pp 92–100Google Scholar
  28. 28.
    Joachims T: Transductive inference for text classification using support vector machines. In: International Conference on Machine Learning, 1999, pp 200–209Google Scholar
  29. 29.
    Pan SJ, Yang Q: A survey on transfer learning. IEEE Trans Knowl Data Eng 22 (10): 1345–1359, 2010CrossRefGoogle Scholar
  30. 30.
    Lawrence ND, Platt JC: Learning to learn with the informative vector machine. In: International Conference on Machine Learning, 2004, pp 65–65Google Scholar
  31. 31.
    Bonilla EV, Chai KM, Williams C: Multi-task gaussian process prediction. Annual Conference on Neural Information Processing Systems, 2007Google Scholar
  32. 32.
    Schwaighofer A, Tresp V, Yu K: Learning gaussian process kernels via hierarchical Bayes. Annual Conference on Neural Information Processing Systems, 2004Google Scholar
  33. 33.
    Evgeniou T, Pontil M: Regularized multi–task learning, 2004, pp 109–117Google Scholar
  34. 34.
    Gao J, Fan W, Jiang J, Han J: Knowledge transfer via multiple model local structure mapping. ACM Knowl Discov Data Mining, 283–291, 2008Google Scholar
  35. 35.
    Glorot X, Bengio Y: Understanding the difficulty of training deep feedforward neural networks 9,249–256, 2010Google Scholar
  36. 36.
    Wanqing C, Rongshou Z, Baade PD, Siwei Z, Hongmei Z, Freddie B, Ahmedin J, Qin YX, Jie H: Cancer statistics in China, 2015, 2016Google Scholar
  37. 37.
    Siegel R, Ma J, Zou Z, Jemal A: Cancer statistics, 2014. CA: Cancer J Clin 64 (1): 9–29, 2014Google Scholar
  38. 38.
    Henschke CI: Early lung cancer action project: overall design and findings from baseline screening 89, 2474–2482, 2000Google Scholar

Copyright information

© Society for Imaging Informatics in Medicine 2019

Authors and Affiliations

  1. 1.School of Information Science and EngineeringShandong UniversityJinanChina
  2. 2.Orthopedics DepartmentTaian Traditional Chinese Medicine HospitalTaianChina
  3. 3.Department of Medical ImagingUniversity of Saskatchewan and Saskatoon Health RegionSaskatoonCanada
  4. 4.Radiology DepartmentThe Second Hospital of Shandong UniversityJinanChina

Personalised recommendations