Performance of a Deep Neural Network Algorithm Based on a Small Medical Image Dataset: Incremental Impact of 3D-to-2D Reformation Combined with Novel Data Augmentation, Photometric Conversion, or Transfer Learning

  • Vikash Gupta
  • Mutlu Demirer
  • Matthew Bigelow
  • Kevin J. Little
  • Sema Candemir
  • Luciano M. Prevedello
  • Richard D. White
  • Thomas P. O’Donnell
  • Michael Wels
  • Barbaros S. ErdalEmail author
Original Paper


Collecting and curating large medical-image datasets for deep neural network (DNN) algorithm development is typically difficult and resource-intensive. While transfer learning (TL) decreases reliance on large data collections, current TL implementations are tailored to two-dimensional (2D) datasets, limiting applicability to volumetric imaging (e.g., computed tomography). Targeting performance enhancement of a DNN algorithm based on a small image dataset, we assessed incremental impact of 3D-to-2D projection methods, one supporting novel data augmentation (DA); photometric grayscale-to-color conversion (GCC); and/or TL on training of an algorithm from a small coronary computed tomography angiography (CCTA) dataset (200 examinations, 50% with atherosclerosis and 50% atherosclerosis-free) producing 245 diseased and 1127 normal coronary arteries/branches. Volumetric CCTA data was converted to a 2D format creating both an Aggregate Projection View (APV) and a Mosaic Projection View (MPV), supporting DA per vessel; both grayscale and color-mapped versions of each view were also obtained. Training was performed both without and with TL, and algorithm performance of all permutations was compared using area under the receiver operating characteristics curve. Without TL, APV performance was 0.74 and 0.87 on grayscale and color images, respectively, compared to 0.90 and 0.87 for MPV. With TL, APV performance was 0.78 and 0.88 on grayscale and color images, respectively, compared with 0.93 and 0.91 for MPV. In conclusion, TL enhances performance of a DNN algorithm from a small volumetric dataset after proposed 3D-to-2D reformatting, but additive gain is achieved with application of either GCC to APV or the proposed novel MPV technique for DA.


Deep neural network Data augmentation Photometric conversion Transfer learning Coronary artery computed tomography angiography Artificial intelligence Medical imaging 


Funding Information

This project was partially funded by:

1. Donation from the Edward J. DeBartolo, Jr. Family.

2. Master Research Agreement with Siemens Healthineers.

Compliance with Ethical Standards

All CCTA image-dataset utilization was retrospective, performed locally under Institutional Review Board approval (including HIPAA compliance) with the waiver of patient consent.


  1. 1.
    Krizhevsky A, Sutskever I, Hinton GE: ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th international conference on neural information processing systems 1, 2012, pp. 1097–1105Google Scholar
  2. 2.
    Perez L, Wang J: The effectiveness of data augmentation in image classification using deep learning. CoRR abs, arXiv:1712.04621, 2017.Google Scholar
  3. 3.
    Prasad S, Kumar P, Sinha KP: Grayscale to color map transformation for efficient image analysis on low processing devices. In: El-Alfy ES., Thampi S., Takagi H., Piramuthu S., Hanne T. (eds) Advances in Intelligent Informatics. Advances in Intelligent Systems and Computing, vol 320. Springer, cham, 2015, Google Scholar
  4. 4.
    Ng HW, Nguyen VD, Vonikakis V, Winkler S: Deep learning for emotion recognition on small datasets using transfer learning. Proceedings of ACM International conference on multimodal interaction, 443–449, 2015.Google Scholar
  5. 5.
    Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM: Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35:1285–1298, 2016CrossRefGoogle Scholar
  6. 6.
    Su H, Maji S, Kalogerakis E, Learned-Miller E: Multi-view convolutional neural networks for 3D shape recognition. IEEE International Conference on Computer Vision (ICCV), 945–953, 2015.Google Scholar
  7. 7.
    Kanezaki A, Matsushita Y, Nishida Y: RotationNet: joint object categorization and pose estimation using multi views from unsupervised viewpoints. International Conference on Computer Vision and Pattern Recognition (CVPR), arXiv:1603.06208, 2018.Google Scholar
  8. 8.
    White RD, Erdal BS, Bigelow MT, Demirer M, Galizia MS, Gupta V, Carpenter JL, Candemir S, Dikici E, O'Donnell TP, Halabi AH, Prevedello LM et al. Augmented intelligence to facilitate exclusion of coronary atherosclerosis on CCTA during emergency department chest-pain presentations: algorithm development. Radiology, Cardiothoracic Imaging (Submitted).Google Scholar
  9. 9.
    Litt HI, Gatsonis C, Snyder B, Singh H, Miller CD, Entrikin DW, Leaming JM, Gavin LJ, Pacella CB, Hollander JE: CT angiography for safe discharge of patients with possible acute coronary syndromes. N Engl J Med 366:1393–1403, 2012CrossRefGoogle Scholar
  10. 10.
    Ghoshhajra BB, Engel LC, Major GP, Goehler A, Techasith T, Verdini D, Do S, Liu B, Li X, Sala M, Kim MS, Blankstein R, Prakash P, Sidhu MS, Corsini E, Banerji D, Wu D, Abbara S, Truong Q, Brady TJ, Hoffmann U, Kalra M: Evolution of coronary computed tomography radiation dose reduction at a tertiary referral center. Am J Med 125:764–772, 2012CrossRefGoogle Scholar
  11. 11.
    Lubbers MM, Dedic A, Kurata A, Dijkshoorn M, Schaap J, Lammers J, Lamfers EJ, Rensing BJ, Braam RL, Nathoe HM, Post JC, Rood PP, Schultz CJ, Moelker A, Ouhlous M, van Dalen BM, Boersma E, Nieman K: Round-the-clock performance of coronary CT angiography for suspected acute coronary syndrome: results from the BEACON trial. Eur Radiol 28:2169–2175, 2018CrossRefGoogle Scholar
  12. 12.
    Demirer M, Candemir S, Bigelow M, Yu SM, Gupta V, Prevedello LM et al.: A user interface for optimizing radiologist engagement in image-data curation for artificial intelligence. Radiology, 2019 (In Press)Google Scholar
  13. 13.
    Siemens Healthineers, Erlangen, Germany. Available at Accessed 14 April 2018
  14. 14.
  15. 15.
    Zheng Y, Tek H, Funka-Lea G: Robust and accurate coronary artery centerline extraction in CTA by combining model-driven and data-driven approaches. Med Image Comput Comput Assist Interv 16:74–81, 2013PubMedGoogle Scholar
  16. 16.
    Dalrymple NC, Prasad SR, Freckleton MW, Chintapalli KN: Informatics in radiology (infoRAD): Introduction to the language of three-dimensional imaging with multidetector CT. RadioGraphics 25:1409–1428, 2005. CrossRefPubMedGoogle Scholar
  17. 17.
    Dobbin KK, Simon RM: Optimally splitting cases for training and testing high dimensional classifiers. BMC Med Genomics 4(31), 2011.
  18. 18.
    ArcMap: Understanding the mosaicking rules for a mosaic dataset. ESRI. Available at Accessed 18 November 2018.
  19. 19.
    Zickler T, Mallick SP, Kriegman DJ, Belhumeur PN: Color subspaces as photometric invariants. International Journal of Computer Vision 79:13–30, 2008CrossRefGoogle Scholar
  20. 20.
    Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z: Rethinking the inception architecture for computer vision. Proceedings of the IEEE conference on computer vision and pattern recognition. 2818–2826. arXiv:1512.00567, 2016.Google Scholar
  21. 21.
    Dahl GE, Sainath TN, Hinton GE: Improving deep neural networks for LVCSR using rectified linear units and dropout. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 8609–8613, 2013.Google Scholar
  22. 22.
    Chollet F: User experience design for APIs. The Keras Blog; Available at Accessed 18 November 2018.
  23. 23.
    Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M, Kudlur M, Levenberg J, Monga R, Moore S, Murray DG, Steiner B, Tucker P, Vasudevan V, Warden P, Wicke M, Yu Y, Zheng X: TensorFlow: a system for large-scale machine learning. USENIX symposium on operating systems design and implementation 16:265–283, 2016. arXiv:1605.08695Google Scholar
  24. 24.
    Bottou L: Large-scale machine learning with stochastic gradient descent. Proceedings of COMPSTAT 177–186, 2010.Google Scholar
  25. 25.
    Zreik M, van Hamersvelt RW, Wolterink JM, Leiner T, Viergever MA, Isgum I: Automatic detection and characterization of coronary artery plaque and stenosis using a recurrent convolutional neural network in coronary CT angiography. Available at arXiv:1804.04360v1. Accessed 18 November 2018.
  26. 26.
    Loewke KE, Camarillo DB, Jobst CA, Salisbury JK: Real-time image mosaicing for medical applications. Stud Health Technol Inform 125:304–309, 2007PubMedGoogle Scholar

Copyright information

© Society for Imaging Informatics in Medicine 2019

Authors and Affiliations

  • Vikash Gupta
    • 1
  • Mutlu Demirer
    • 1
  • Matthew Bigelow
    • 1
  • Kevin J. Little
    • 1
  • Sema Candemir
    • 1
  • Luciano M. Prevedello
    • 1
  • Richard D. White
    • 1
  • Thomas P. O’Donnell
    • 2
  • Michael Wels
    • 3
  • Barbaros S. Erdal
    • 1
    Email author
  1. 1.Laboratory for Augmented Intelligence in Imaging-Division of Medical Imaging Informatics, Department of RadiologyOhio State University College of MedicineColumbusUSA
  2. 2.Siemens HealthineersMalvernUSA
  3. 3.Siemens HealthineersErlangenGermany

Personalised recommendations