Multimedia Tools and Applications

, Volume 78, Issue 22, pp 31515–31532 | Cite as

Automated TB classification using ensemble of deep architectures

  • Rahul Hooda
  • Ajay MittalEmail author
  • Sanjeev Sofat


Tuberculosis (TB) is an infectious disease that mainly affects the lung region. Its initial screening is mostly performed using chest radiograph, which is also recommended by the World Health Organization. To help the radiologists in diagnosing this disease, different computer-aided diagnosis (CAD) systems have been developed. However, the development of these systems are still in the early phases as it is extremely challenging to automatically detect TB. This is due to extreme variations in the impact caused by TB on the CXR. In this study, a deep-learning-based TB detection system has been presented which achieves significantly high accuracy. The proposed method is an ensemble of three standard architectures namely AlexNet, GoogleNet and ResNet. The significant contribution of the study is to train these architectures from scratch and creating an ensemble suited to perform TB classification. The proposed method is trained and evaluated on a combined dataset formed using publicly available standard datasets. The ensemble attains the accuracy of 88.24% and area under the curve is equal to 0.93, which eclipses the performance of most of the existing methods.


Tuberculosis classification Medical image analysis Deep-learning Chest radiograph 



  1. 1.
    Arzhaeva Y, Hogeweg L, de Jong P, Viergever M, van Ginneken B (2009) Global and local multi-valued dissimilarity-based classification: application to computer-aided detection of tuberculosis. Med Image Comput Computer-Assisted Intervention–MICCAI 2009:724–731Google Scholar
  2. 2.
    Bar Y, Diamant I, Wolf L, Lieberman S, Konen E, Greenspan H (2015) Chest pathology detection using deep learning with non-medical training. In: 2015 IEEE 12th international symposium on biomedical imaging (ISBI), pp 294–297Google Scholar
  3. 3.
    Bar Y, Diamant I, Wolf L, Lieberman S, Konen E, Greenspan H (2018) Chest pathology identification using deep feature selection with non-medical training. Comput Methods Biomechanics Biomedical Eng Image Visualization 6(3):259–263CrossRefGoogle Scholar
  4. 4.
  5. 5.
    Cao Y, Liu C, Liu B, Brunette MJ, Zhang N, Sun T, Zhang P, Peinado J, Garavito ES, Garcia LL et al (2016) Improving tuberculosis diagnostics using deep learning and mobile health technologies among resource-poor and marginalized communities. In: 2016 IEEE first international conference on connected health: applications, systems and engineering technologies (CHASE), pp 274–281Google Scholar
  6. 6.
    Chen H, Dou Q, Wang X, Qin J, Heng PA, et al. (2016) Mitosis detection in breast cancer histology images via deep cascaded networks. In: AAAI, pp 1160–1166Google Scholar
  7. 7.
    Cireşan DC, Giusti A, Gambardella LM, Schmidhuber J (2013) Mitosis detection in breast cancer histology images with deep neural networks. In: International conference on medical image computing and computer-assisted intervention, pp 411–418CrossRefGoogle Scholar
  8. 8.
    Ebrahimian H, Rijal OM, Noor NM, Yunus A, Mahyuddin AA (2014) Phase congruency parameter estimation and discrimination ability in detecting lung disease chest radiograph. In: 2014 IEEE conference on biomedical engineering and sciences (IECBES), pp 729–734Google Scholar
  9. 9.
    Gao L, Guo Z, Zhang H, Xu X, Shen H (2017) Video captioning with attention-based lstm and semantic consistency. IEEE Trans Multimedia 19(9):2045–2055CrossRefGoogle Scholar
  10. 10.
    van Ginneken B, Katsuragawa S, ter Haar Romeny BM, Doi K, Viergever MA (2002) Automatic detection of abnormalities in chest radiographs using local texture analysis. IEEE Trans Med Imaging 21(2):139–149CrossRefGoogle Scholar
  11. 11.
    Girardi D, Küng J, Kleiser R, Sonnberger M, Csillag D, Trenkler J, Holzinger A (2016) Interactive knowledge discovery with the doctor-in-the-loop: a practical example of cerebral aneurysms research. Brain Inform 3(3):133CrossRefGoogle Scholar
  12. 12.
    Havaei M, Davy A, Warde-Farley D, Biard A, Courville A, Bengio Y, Pal C, Jodoin PM, Larochelle H (2017) Brain tumor segmentation with deep neural networks. Med Image Anal 35:18–31CrossRefGoogle Scholar
  13. 13.
    He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. arXiv:1512.03385
  14. 14.
    Heo SJ, Kim Y, Yun S, Lim SS, Kim J, Nam CM, Park EC, Jung I, Yoon JH (2019) Deep learning algorithms with demographic information help to detect tuberculosis in chest radiographs in annual workers’ health examination data. Int J Environ Res Public Health 16(2):250CrossRefGoogle Scholar
  15. 15.
    Hogeweg L, Mol C, de Jong PA, Dawson R, Ayles H, van Ginneken B (2010) Fusion of local and global detection systems to detect tuberculosis in chest radiographs. In: International conference on medical image computing and computer-assisted intervention, pp 650–657Google Scholar
  16. 16.
    Hogeweg L, Sánchez CI, Maduskar P, Philipsen R, Story A, Dawson R, Theron G, Dheda K, Peters-Bax L, van Ginneken B (2015) Automatic detection of tuberculosis in chest radiographs using a combination of textural, focal, and shape abnormality analysis. IEEE Trans Med Imaging 34(12):2429–2442CrossRefGoogle Scholar
  17. 17.
    Holzinger A (2016) Interactive machine learning for health informatics: when do we need the human-in-the-loop? Brain Inform 3(2):119–131CrossRefGoogle Scholar
  18. 18.
    Holzinger A, Plass M, Holzinger K, Crişan GC, Pintea CM, Palade V (2016) Towards interactive machine learning (iml): applying ant colony algorithms to solve the traveling salesman problem with the human-in-the-loop approach. In: International conference on availability, reliability, and security. Springer, pp 81–95Google Scholar
  19. 19.
    Huang G, Liu Z, Weinberger KQ, van der Maaten L (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, p 3Google Scholar
  20. 20.
    Hwang S, Kim HE, Jeong J, Kim HJ (2016) A novel approach for tuberculosis screening based on deep convolutional neural networks. In: SPIE medical imaging, pp 97,852w–97,852wGoogle Scholar
  21. 21.
    Jaeger S, Karargyris A, Antani S, Thoma G (2012) Detecting tuberculosis in radiographs using combined lung masks. In: 2012 Annual international conference of the IEEE engineering in medicine and biology society, pp 4978–4981Google Scholar
  22. 22.
    Jaeger S, Candemir S, Antani S, Wang YX, Lu PX, Thoma G (2014) Two public chest X-ray datasets for computer-aided screening of pulmonary diseases. Quant Imaging Med Surg 4(6):475–477Google Scholar
  23. 23.
    Jaeger S, Karargyris A, Candemir S, Folio L, Siegelman J, Callaghan F, Xue Z, Palaniappan K, Singh RK, Antani S et al (2014) Automatic tuberculosis screening using chest radiographs. IEEE Trans Med Imaging 33(2):233–245CrossRefGoogle Scholar
  24. 24.
    Jagoe JR, Paton KA (1976) Measurement of pneumoconiosis by computer. IEEE Trans Comput C-25(1):95–97CrossRefGoogle Scholar
  25. 25.
    Karargyris A, Siegelman J, Tzortzis D, Jaeger S, Candemir S, Xue Z, Santosh K, Vajda S, Antani S, Folio L et al (2016) Combination of texture and shape features to detect pulmonary abnormalities in digital chest x-rays. Int J Comput Assist Radiol Surg 11(1):99–106CrossRefGoogle Scholar
  26. 26.
    Koeslag A, de Jager G (2001) Computer aided diagnosis of miliary tuberculosis. In: Proceedings of the pattern recognition association of South AfricaGoogle Scholar
  27. 27.
    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105Google Scholar
  28. 28.
    Lakhani P, Sundaram B (2017) Deep learning at chest radiography: automated classification of pulmonary tuberculosis by using convolutional neural networks. Radiology 284(2):574–582CrossRefGoogle Scholar
  29. 29.
    Le K (2006) Automated detection of early lung cancer and tuberculosis based on x-ray image analysis. In: Proceedings WSEAS international conference on signal, speech and image processing, pp 1–6Google Scholar
  30. 30.
    LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324CrossRefGoogle Scholar
  31. 31.
    Maduskar P, Philipsen R, Melendez J, Scholten E, Chanda D, Ayles H, Sánchez CI, van Ginneken B (2016) Automatic detection of pleural effusion in chest radiographs. Med Image Analysis 28:22–32CrossRefGoogle Scholar
  32. 32.
    Melendez J, Sánchez C, Philipsen R, Maduskar P, van Ginneken B (2014) Multiple-instance learning for computer-aided detection of tuberculosis. In: SPIE medical imaging, pp 90,351j–90,351jGoogle Scholar
  33. 33.
    Melendez J, van Ginneken B, Maduskar P, Philipsen R, Ayles H, Sánchez CI (2016) On combining multiple-instance learning and active learning for computer-aided detection of tuberculosis. IEEE Trans Med Imaging 35(4):1013–1024CrossRefGoogle Scholar
  34. 34.
    Organization WH (2016) Global tuberculosis report 2016Google Scholar
  35. 35.
    Paul JL, Levine MD, Fraser RG, Laszlo CA (1974) The measurement of total lung capacity based on a computer analysis of anterior and lateral radiographic chest images. IEEE Trans Biomed Eng BME-21(6):444–451CrossRefGoogle Scholar
  36. 36.
    Pereira S, Pinto A, Alves V, Silva CA (2016) Brain tumor segmentation using convolutional neural networks in mri images. IEEE Trans Med Imaging 35(5):1240–1251CrossRefGoogle Scholar
  37. 37.
    Prasoon A, Petersen K, Igel C, Lauze F, Dam E, Nielsen M (2013) Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network. In: International conference on medical image computing and computer-assisted intervention, pp 246–253Google Scholar
  38. 38.
    Ratnasari N, Susanto A, Soesanti I, et al. (2013) Thoracic x-ray features extraction using thresholding-based roi template and pca-based features selection for lung tb classification purposes. In: 2013 3rd international conference on instrumentation, communications, information technology, and biomedical engineering (ICICI-BME), pp 65–69Google Scholar
  39. 39.
    Rijal OM, Ebrahimian H, Noor NM (2012) Determining features for discriminating ptb and normal lungs using phase congruency model. In: Proceedings of 2012 IEEE-EMBS international conference on biomedical and health informatics, pp 341–344Google Scholar
  40. 40.
    Roth HR, Farag A, Lu L, Turkbey EB, Summers RM (2015) Deep convolutional networks for pancreas segmentation in ct imaging. In: Medical imaging 2015: image processing, vol 9413, p 94131gGoogle Scholar
  41. 41.
    Roth HR, Lu L, Farag A, Shin HC, Liu J, Turkbey EB, Summers RM (2015) Deeporgan: multi-level deep convolutional networks for automated pancreas segmentation. In: International conference on medical image computing and computer-assisted intervention, pp 556–564Google Scholar
  42. 42.
    Shen R, Cheng I, Basu A (2010) A hybrid knowledge-guided detection technique for screening of infectious pulmonary tuberculosis from chest radiographs. IEEE Trans Biomed Eng 57(11):2646–2656CrossRefGoogle Scholar
  43. 43.
    Shiraishi J, Katsuragawa S, Ikezoe J, Matsumoto T, Kobayashi T, Komatsu K, Matsui M, Fujita H, Kodera Y, Doi K (2000) Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists’ detection of pulmonary nodules. AJR Am J Roentgenol 174(1):71–74CrossRefGoogle Scholar
  44. 44.
    Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556
  45. 45.
    Song J, Gao L, Nie F, Shen H, Yan Y, Sebe N (2016) Optimized graph learning using partial tags and multiple features for image and video annotation. IEEE Trans Image Process 25(11):4999–5011MathSciNetzbMATHCrossRefGoogle Scholar
  46. 46.
    Song J, Guo Y, Gao L, Li X, Hanjalic A, Shen H (2018) From deterministic to generative: multimodal stochastic rnns for video captioning. IEEE Trans Neural Netw Learning Sys (99):1–12Google Scholar
  47. 47.
    Song J, Zhang H, Li X, Gao L, Wang M, Hong R (2018) Self-supervised video hashing with hierarchical binary auto-encoder. IEEE Trans Image Process 27 (7):3210–3221MathSciNetzbMATHCrossRefGoogle Scholar
  48. 48.
    Sutton RN, Hall EL (1972) Texture measures for automatic classification of pulmonary disease. IEEE Trans Comput C-21(7):667–676CrossRefGoogle Scholar
  49. 49.
    Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9Google Scholar
  50. 50.
    Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826Google Scholar
  51. 51.
    Tan JH, Acharya UR, Tan C, Abraham KT, Lim CM (2012) Computer-assisted diagnosis of tuberculosis: a first order statistical approach to chest radiograph. J Med Sys 36(5):2751–2759CrossRefGoogle Scholar
  52. 52.
    Wang H, Roa AC, Basavanhally AN, Gilmore HL, Shih N, Feldman M, Tomaszewski J, Gonzalez F, Madabhushi A (2014) Mitosis detection in breast cancer pathology images by combining handcrafted and convolutional neural network features. J Med Imaging 1(3):034,003CrossRefGoogle Scholar
  53. 53.
    Wang X, Gao L, Song J, Shen H (2017) Beyond frame-level cnn: saliency-aware 3-d cnn with lstm for video action recognition. IEEE Signal Process Lett 24(4):510–514CrossRefGoogle Scholar
  54. 54.
    Wang X, Gao L, Wang P, Sun X, Liu X (2018) Two-stream 3-d convnet fusion for action recognition in videos with arbitrary size and length. IEEE Trans Multimedia 20(3):634–644CrossRefGoogle Scholar
  55. 55.
    Xu T, Cheng I, Long R, Mandal M (2013) Novel coarse-to-fine dual scale technique for tuberculosis cavity detection in chest radiographs. EURASIP J Image Video Process 2013(1):1CrossRefGoogle Scholar
  56. 56.
    Yan C, Xie H, Chen J, Zha Z, Hao X, Zhang Y, Dai Q (2018) A fast uyghur text detector for complex background images. IEEE Trans Multimedia 20 (12):3389–3398CrossRefGoogle Scholar
  57. 57.
    Yan C, Li L, Zhang C, Liu B, Zhang Y, Dai Q (2019) Cross-modality bridging and knowledge transferring for image understanding. IEEE Trans MultimediaGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer Science and EngineeringPunjab Engineering College (Deemed to be University)ChandigarhIndia
  2. 2.UIETPanjab UniversityChandigarhIndia

Personalised recommendations