Journal of Medical Systems

, 42:231 | Cite as

Generative Adversarial Network for Medical Images (MI-GAN)

  • Talha Iqbal
  • Hazrat AliEmail author
Image & Signal Processing
Part of the following topical collections:
  1. Advanced Computational Intelligence and Soft Computing in Medical Imaging


Deep learning algorithms produces state-of-the-art results for different machine learning and computer vision tasks. To perform well on a given task, these algorithms require large dataset for training. However, deep learning algorithms lack generalization and suffer from over-fitting whenever trained on small dataset, especially when one is dealing with medical images. For supervised image analysis in medical imaging, having image data along with their corresponding annotated ground-truths is costly as well as time consuming since annotations of the data is done by medical experts manually. In this paper, we propose a new Generative Adversarial Network for Medical Imaging (MI-GAN). The MI-GAN generates synthetic medical images and their segmented masks, which can then be used for the application of supervised analysis of medical images. Particularly, we present MI-GAN for synthesis of retinal images. The proposed method generates precise segmented images better than the existing techniques. The proposed model achieves a dice coefficient of 0.837 on STARE dataset and 0.832 on DRIVE dataset which is state-of-the-art performance on both the datasets.


GAN Medical imaging Style transfer Deep learning Retinal images 



No funding declared.

Compliance with Ethical Standards

Conflict of interests

Talha Iqbal declares that he has no conflict of interest. Hazrat Ali declares that he has no conflict of interest.


  1. 1.
    de Vos, B.D., Wolterink, J.M., de Jong, P.A., Viergever, M.A., and Išgum, I.: 2D image classification for 3D anatomy localization: employing deep convolutional neural networks. In: Medical imaging 2016: Image processing, vol. 9784, international society for optics and photonics, 2016.Google Scholar
  2. 2.
    Cai, Y., Landis, M., Laidley, D.T., Kornecki, A., Lum, A., and Li, S., Multi-modal vertebrae recognition using transformed deep convolution network. Comput. Med. Imaging Graph. 51:11–19, 2016.CrossRefGoogle Scholar
  3. 3.
    Chen, H., Ni, D., Qin, J., Li, S., Yang, X., Wang, T., and Heng, P.A., Standard plane localization in fetal ultrasound via domain transferred deep neural networks. IEEE J. of Biomedical and Health Informatics 19 (5):1627–1636, 2015.CrossRefGoogle Scholar
  4. 4.
    Kumar, A., Sridar, P., Quinton, A., Kumar, R.K., Feng, D., Nanan, R., and Kim, J.: Plane identification in fetal ultrasound images using saliency maps and convolutional neural networks. In: Biomedical Imaging (ISBI), 2016 IEEE 13th International Symposium on, IEEE, pp. 791–794, 2016.Google Scholar
  5. 5.
    Ghesu, F.C., Georgescu, B., Mansi, T., Neumann, D., Hornegger, J., and Comaniciu, D.: An artificial agent for anatomical landmark detection in medical images. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, pp. 229–237, 2016.Google Scholar
  6. 6.
    Baumgartner, C.F., Kamnitsas, K., Matthew, J., Smith, S., Kainz, B., and Rueckert, D.: Real-time standard scan plane detection and localisation in fetal ultrasound using fully convolutional neural networks. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, pp. 203–211, 2016.Google Scholar
  7. 7.
    Kong, B., Zhan, Y., Shin, M., Denny, T., and Zhang, S.: Recognizing end-diastole and end-systole frames via deep temporal regression network. In: International conference on medical image computing and computer-assisted intervention, Springer, pp. 264–272, 2016.Google Scholar
  8. 8.
    Barbu, A., Lu, L., Roth, H., Seff, A., and Summers, R.M., An analysis of robust cost functions for cnn in computer-aided diagnosis. Comput. Methods Biomechanics Biomedical Eng.: Imaging & Vis. 6(3):253–258, 2018.Google Scholar
  9. 9.
    Roth, H.R., Lu, L., Liu, J., Yao, J., Seff, A., Cherry, K., Kim, L., and Summers, R.M., Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans. Med. Imaging 35(5):1170–1181, 2016.CrossRefGoogle Scholar
  10. 10.
    Teramoto, A., Fujita, H., Yamamuro, O., and Tamaki, T., Automated detection of pulmonary nodules in PET/CT images: Ensemble false-positive reduction using a convolutional neural network technique. Med. Phys. 43(6Part1):2821–2827, 2016.CrossRefGoogle Scholar
  11. 11.
    Becker, C., Rigamonti, R., Lepetit, V., and Fua, P.: Supervised feature learning for curvilinear structure segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, pp. 526–533, 2013.Google Scholar
  12. 12.
    Makhzani, A., and Frey, B.J.: PixelGAN autoencoders. In: Advances in Neural Information Processing Systems, pp. 1972–1982, 2017.Google Scholar
  13. 13.
    Ganin, Y., and Lempitsky, V.: N4-Fields: Neural Network Nearest Neighbor Fields for Image Transforms. In: Asian Conference on Computer Vision, Springer, pp. 536–551, 2014.Google Scholar
  14. 14.
    Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., van der Laak, J.A., van Ginneken, B., and Sánchez, C.I., A survey on deep learning in medical image analysis. Med. Image Anal. 42:60–88, 2017.CrossRefGoogle Scholar
  15. 15.
    Gatys, L.A., Ecker, A.S., and Bethge, M.: A neural algorithm of artistic style, arXiv:1508.06576
  16. 16.
    Ulyanov, D., Lebedev, V., Vedaldi, A., and Lempitsky, V.S.. In: ICML, pp. 1349–1357, 2016.Google Scholar
  17. 17.
    Johnson, J., Alahi, A., and Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: European Conference on Computer Vision, Springer, pp. 694–711, 2016.Google Scholar
  18. 18.
    Kirbas, C., and Quek, F., A review of vessel extraction techniques and algorithms. ACM Comput. Surveys (CSUR) 36(2):81–121, 2004.CrossRefGoogle Scholar
  19. 19.
    Lesage, D., Angelini, E.D., Bloch, I., and Funka-Lea, G., A review of 3D vessel lumen segmentation techniques: models, features and extraction schemes. Med. Image Anal. 13(6):819–845, 2009.CrossRefGoogle Scholar
  20. 20.
    Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y.: Generative adversarial nets. In: Advances in neural information processing systems, pp. 2672–2680, 2014.Google Scholar
  21. 21.
    Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A.A.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: AAAI, Vol. 4, p. 12, 2017.Google Scholar
  22. 22.
    Ding, C., Xia, Y., and Li, Y.: Supervised segmentation of vasculature in retinal images using neural networks. In: Orange technologies (ICOT), 2014 IEEE International Conference on, IEEE, pp. 49–52, 2014.Google Scholar
  23. 23.
    Radford, A., Metz, L., and Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks, arXiv:1511.06434
  24. 24.
    Mirza, M., and Osindero, S.: Conditional generative adversarial nets, arXiv:1411.1784
  25. 25.
    Denton, E.L., Chintala, S., Fergus, R., et al.: Deep generative image models using a Laplacian pyramid of adversarial networks. In: Advances in Neural Information Processing Systems, pp. 1486–1494, 2015.Google Scholar
  26. 26.
    Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., and Courville, A.C.: Improved training of Wasserstein GANs. In: Advances in Neural Information Processing Systems, pp. 5769–5779, 2017.Google Scholar
  27. 27.
    Peng, H., Hawrylycz, M., Roskams, J., Hill, S., Spruston, N., Meijering, E., and Ascoli, G.A., Bigneuron: large-scale 3d neuron reconstruction from optical microscopy images. Neuron 87(2):252–256, 2015.CrossRefGoogle Scholar
  28. 28.
    Isola, P., Zhu, J.-Y., Zhou, T., and Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5967–5976, 2017.Google Scholar
  29. 29.
    Zhang, J., Chen, L., Zhuo, L., Liang, X., and Li, J., An Efficient Hyperspectral Image Retrieval Method: Deep Spectral-Spatial Feature Extraction with DCGAN and Dimensionality Reduction Using t-SNE-based NM Hashing. Remote Sens. 10(2):271, 2018.CrossRefGoogle Scholar
  30. 30.
    Wan, L., Zeiler, M., Zhang, S., Le Cun, Y., and Fergus, R.: Regularization of neural networks using dropconnect. In: International Conference on Machine Learning, pp. 1058–1066, 2013.Google Scholar
  31. 31.
    Wang, X., and Gupta, A.: Generative image modeling using style and structure adversarial networks. In: European Conference on Computer Vision, Springer, pp. 318–335, 2016.Google Scholar
  32. 32.
    Mao, X., Shen, C., and Yang, Y.-B.: Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In: Advances in neural information processing systems, pp. 2802–2810, 2016.Google Scholar
  33. 33.
    Zhu, J.-Y., Park, T., Isola, P., and Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2242–2251, 2017.Google Scholar
  34. 34.
    Ronneberger, O., Fischer, P., and Brox, T.: U-NET: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention, Springer, pp. 234–241, 2015.Google Scholar
  35. 35.
    Zhao, H., Li, H., and Cheng, L.: Synthesizing Filamentary Structured Images with GANs, arXiv:1706.02185
  36. 36.
    Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A.L., Deeplab: Semantic image segmentation with deep convolutional nets, Atrous Convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4):834–848, 2018.CrossRefGoogle Scholar
  37. 37.
    Li, Y., Wang, N., Shi, J., Liu, J., and Hou, X., Revisiting batch normalization for practical domain adaptation. Pattern Recogn. 80:109–117, 2018.CrossRefGoogle Scholar
  38. 38.
    Zhang, W., Li, W., Yan, J., Yu, L., and Pan, C., Adaptive threshold selection for background removal in fringe projection profilometry. Opt. Lasers Eng. 90:209–216, 2017.CrossRefGoogle Scholar
  39. 39.
    Son, J., Park, S.J., and Jung, K.-H.: Retinal vessel segmentation in fundoscopic images with generative adversarial networks, arXiv:1706.09318
  40. 40.
    Maninis, K.-K., Pont-Tuset, J., Arbeláez, P., and Van Gool, L.: Deep retinal image understanding. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, pp. 140–148, 2016.Google Scholar
  41. 41.
    Farokhian, F., Yang, C., Demirel, H., Wu, S., and Beheshti, I., Automatic parameters selection of Gabor filters with the imperialism competitive algorithm with application to retinal vessel segmentation. Biocybernetics Biomedical Eng. 37(1):246–254, 2017.CrossRefGoogle Scholar
  42. 42.
    Xie, S., and Tu, Z.: Holistically-nested edge detection. In: Proceedings of the IEEE international conference on computer vision, pp. 1395–1403, 2015.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Electrical EngineeringCOMSATS University Islamabad, Abbottabad CampusIslamabadPakistan

Personalised recommendations