Abstract
Data augmentation is an important strategy for enlarging training datasets in deep learning-based medical image analysis. This is because large, annotated medical datasets are not only difficult and costly to generate, but also quickly become obsolete due to rapid advances in imaging technology. Image-to-image conditional generative adversarial networks (C-GAN) provide a potential solution for data augmentation. However, annotations used as inputs to C-GAN are typically based only on shape information, which can result in undesirable intensity distributions in the resulting artificially-created images. In this paper, we introduce an active cell appearance model (ACAM) that can measure statistical distributions of shape and intensity and use this ACAM model to guide C-GAN to generate more realistic images, which we call A-GAN. A-GAN provides an effective means for conveying anisotropic intensity information to C-GAN. A-GAN incorporates a statistical model (ACAM) to determine how transformations are applied for data augmentation. Traditional approaches for data augmentation that are based on arbitrary transformations might lead to unrealistic shape variations in an augmented dataset that are not representative of real data. A-GAN is designed to ameliorate this. To validate the effectiveness of using A-GAN for data augmentation, we assessed its performance on cell analysis in adaptive optics retinal imaging, which is a rapidly-changing medical imaging modality. Compared to C-GAN, A-GAN achieved stability in fewer iterations. The cell detection and segmentation accuracy when assisted by A-GAN augmentation was higher than that achieved with C-GAN. These findings demonstrate the potential for A-GAN to substantially improve existing data augmentation methods in medical image analysis.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Armanious, K., Yang, C., Fischer, M., et al.: MedGAN: medical image translation using GANs. CoRR abs/1806.06397 (2018)
Chi, Y., Bi, L., Kim, J., Feng, D., Kumar, A.: Controlled synthesis of dermoscopic images via a new color labeled generative style transfer network to enhance melanoma segmentation. In: Proceedings of the IEEE Conference on Engineering in Medicine and Biology Society, pp. 2591–2594 (2018)
Cootes, T., Edwards, G., Taylor, C.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 681–685 (2001)
Frid-Adar, M., Diamant, J., Klang, E., et al.: GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing 321, 321–331 (2018)
Goodfellow, I., Pouget-Abadie, J., Mirae, M., et al.: Generative adversarial networks. In: NIPS, pp. 2672–2680 (2014)
Han, Z., Wei, B., Mercado, A., et al.: Spine-GAN: semantic segmentation of multiple spinal structures. Med. Image Anal. 50, 23–35 (2018)
Hussain, Z., Gimenez, F., Yi, D., Rubin, D.: Differential data augmentation techniques for medical imaging classification tasks. In: AMIA Annual Symposium Proceedings, pp. 979–984 (2017)
Iqbal, T., Ali, H.: Generative adversarial network for medical images (MI-GAN). J. Med. Syst. 42(11), 1–11 (2018)
Isola, P., Zhu, J., Zhou, T., Efros, A.: Image-to-image translation with conditional adversarial networks. In: IEEE CVPR (2017)
Liu, S., Gibson, E., Grbic, S., et al.: Decompose to manipulate: manipulable object synthesis in 3D medical images with structured image decomposition. CoRR abs/1812.01737 (2018)
Ramesh, N., Tasdizen, T.: Cell segmentation using a similarity interface with a multi-task convolutional neural network. IEEE J. Biomed. Health Inform. 23(4), 1457–1468 (2019)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Roorda, A., Duncan, J.: Adaptive optics ophthalmoscopy. Ann. Rev. Vis. Sci. 1, 19–50 (2015)
Scoles, D., Sulai, Y., Langlo, C., et al.: In vivo imaging of human cone photoreceptor inner segments. Invest. Ophthalmol. Vis. Sci. 55(7), 4244–4251 (2014)
Wu, E., Wu, K., Cox, D., Lotter, W.: Conditional infilling GANs for data augmentation in mammogram classification. In: Stoyanov, D., et al. (eds.) RAMBO/BIA/TIA-2018. LNCS, vol. 11040, pp. 98–106. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00946-5_11
Zhang, C., Tavanapong, W., Wong, J., de Groen, P.C., Oh, J.H.: Real data augmentation for medical image classification. In: Cardoso, M.J., et al. (eds.) LABELS/CVII/STENT-2017. LNCS, vol. 10552, pp. 67–76. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67534-3_8
Zhang, Z., Yang, L., Zheng, Y.: Translating and segmenting multimodal medical volumes with cycle- and shape-consistency generative adversarial network. In: IEEE CVPR (2018)
Acknowledgments
This work was supported by the Intramural Research Program of the National Institutes of Health, National Eye Institute.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply
About this paper
Cite this paper
Liu, J., Shen, C., Liu, T., Aguilera, N., Tam, J. (2019). Active Appearance Model Induced Generative Adversarial Network for Controlled Data Augmentation. In: Shen, D., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science(), vol 11764. Springer, Cham. https://doi.org/10.1007/978-3-030-32239-7_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-32239-7_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32238-0
Online ISBN: 978-3-030-32239-7
eBook Packages: Computer ScienceComputer Science (R0)
