Adversarial Policy Gradient for Deep Learning Image Augmentation

  • Kaiyang ChengEmail author
  • Claudia Iriondo
  • Francesco Calivá
  • Justin Krogue
  • Sharmila Majumdar
  • Valentina Pedoia
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11769)


The use of semantic segmentation for masking and cropping input images has proven to be a significant aid in medical imaging classification tasks by decreasing the noise and variance of the training dataset. However, implementing this approach with classical methods is challenging: the cost of obtaining a dense segmentation is high, and the precise input area that is most crucial to the classification task is difficult to determine a-priori. We propose a novel joint-training deep reinforcement learning framework for image augmentation. A segmentation network, weakly supervised with policy gradient optimization, acts as an agent, and outputs masks as actions given samples as states, with the goal of maximizing reward signals from the classification network. In this way, the segmentation network learns to mask unimportant imaging features. Our method, Adversarial Policy Gradient Augmentation (APGA), shows promising results on Stanford’s MURA dataset and on a hip fracture classification task with an increase in global accuracy of up to 7.33% and improved performance over baseline methods in 9/10 tasks evaluated. We discuss the broad applicability of our joint training strategy to a variety of medical imaging tasks.


Deep reinforcement learning Adversarial training Semantic segmentation Image augmentation 


  1. 1.
  2. 2.
    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR 2009 (2009)Google Scholar
  3. 3.
    DeVries, T., Taylor, G.W.: Improved regularization of convolutional neural networks with cutout. arXiv:1708.04552 [cs], August 2017
  4. 4.
    Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. arXiv:1608.06993 [cs], August 2016
  5. 5.
    Iglovikov, V., Shvets, A.: TernausNet: U-Net with VGG11 encoder pre-trained on ImageNet for image segmentation. arXiv:1801.05746 [cs], January 2018
  6. 6.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv:1412.6980 [cs], December 2014
  7. 7.
    Rajpurkar, P., et al.: MURA: large dataset for abnormality detection in musculoskeletal radiographs. arXiv:1712.06957 [physics], December 2017
  8. 8.
    Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. arXiv:1409.0575 [cs], September 2014
  9. 9.
    Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. arXiv:1610.02391 [cs], October 2016
  10. 10.
    Wallenberg, M., Forssén, P.: Attentional masking for pre-trained deep networks. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 6149–6154, September 2017.
  11. 11.
    Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8(3), 229–256 (1992). Scholar
  12. 12.
    Zoph, B., Le, Q.V.: Neural architecture search with reinforcement learning. arXiv:1611.01578 [cs], November 2016

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Kaiyang Cheng
    • 1
    • 2
    Email author
  • Claudia Iriondo
    • 1
    • 2
  • Francesco Calivá
    • 2
  • Justin Krogue
    • 3
  • Sharmila Majumdar
    • 2
  • Valentina Pedoia
    • 2
  1. 1.University of CaliforniaBerkeleyUSA
  2. 2.Department of Radiology and Biomedical ImagingUniversity of CaliforniaSan FranciscoUSA
  3. 3.University of CaliforniaSan FranciscoUSA

Personalised recommendations