Averaged Stochastic Optimization for Medical Image Registration Based on Variance Reduction

  • Wei SunEmail author
  • Dirk H. J. Poot
  • Xuan Yang
  • Wiro J. Niessen
  • Stefan Klein
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10883)


In image registration the optimal transformation parameters of a given transformation model are typically obtained by minimizing a cost function. Stochastic gradient descent (SGD) is an efficient optimization algorithm for image registration. In SGD optimization, stochastic approximations of the cost function derivative are used in each iteration to update the transformation parameters. The stochastic approximation error leads to large variance in the parameters. To enforce convergence nonetheless, SGD methods are typically implemented in combination with a gradually decreasing update step size. However, selecting a proper sequence of step sizes is a major challenge in practice. An alternative strategy in numerical optimization is to use a constant step size and enforce convergence by averaging the parameters obtained by SGD over several iterations. It was proven mathematically that the highest possible rate of convergence is achieved in this way. Inspired by this work, we propose an averaged SGD (Avg-SGD) method for efficient image registration. In the Avg-SGD approach, a constant step size is used, in combination with an exponentially weighted iterate averaging scheme. Experiments on 3D lung CT scans demonstrate the effectiveness of the Avg-SGD method in terms of convergence rate, accuracy and precision.


  1. 1.
    Klein, S., Staring, M., Pluim, J.P.W.: Evaluation of optimization methods for nonrigid medical image registration using mutual information and B-splines. IEEE Trans. Image Process. 16(12), 2879–2890 (2007)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Viola, P., Wells III, W.M.: Alignment by maximization of mutual information. Int. J. Comput. Vis. 24(2), 137–154 (1997)CrossRefGoogle Scholar
  3. 3.
    Sun, W., Poot, D.H., Smal, I., Yang, X., Niessen, W.J., Klein, S.: Stochastic optimization with randomized smoothing for image registration. Med. Image Anal. 35, 146–158 (2017)CrossRefGoogle Scholar
  4. 4.
    Sun, W., Niessen, W.J., Klein, S.: Randomly perturbed B-splines for nonrigid image registration. IEEE Trans. Pattern Anal. Mach. Intell. 39(7), 1401–1413 (2017)CrossRefGoogle Scholar
  5. 5.
    Kushner, H.J., Yin, G.: Stochastic Approximation and Recursive Algorithms and Applications, vol. 35. Springer, New York (2003). Scholar
  6. 6.
    Klein, S., Pluim, J.P.W., Staring, M., Viergever, M.A.: Adaptive stochastic gradient descent optimisation for image registration. Int. J. Comput. Vis. 81(3), 227–239 (2009)CrossRefGoogle Scholar
  7. 7.
    Qiao, Y., van Lew, B., Lelieveldt, B.P., Staring, M.: Fast automatic step size estimation for gradient descent optimization of image registration. IEEE Trans. Med. Imaging 35(2), 391–403 (2016)CrossRefGoogle Scholar
  8. 8.
    Bottou, L., Le Cun, Y.: On-line learning for very large data sets. Appl. Stoch. Models. Bus. Ind. 21(2), 137–151 (2005)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Bordes, A., Bottou, L., Gallinari, P.: SGD-QN: careful quasi-Newton stochastic gradient descent. J. Mach. Learn. Res. 10, 1737–1754 (2009)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Xu, W.: Towards optimal one pass large scale learning with averaged stochastic gradient descent. arXiv preprint arXiv:1107.2490 (2011)
  11. 11.
    Polyak, B.T., Juditsky, A.B.: Acceleration of stochastic approximation by averaging. SIAM J. Control Optim. 30(4), 838–855 (1992)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., Suetens, P.: Multimodality image registration by maximization of mutual information. IEEE Trans. Med. Imaging 16(2), 187–198 (1997)CrossRefGoogle Scholar
  13. 13.
    Ruppert, D.: Efficient estimations from a slowly convergent Robbins-Monro process. Technical report, Cornell University Operations Research and Industrial Engineering (1988)Google Scholar
  14. 14.
    Yin, G.: Stochastic approximation via averaging: the Polyak’s approach revisited. In: Pflug, G., Dieter, U. (eds.) Simulation and Optimization. Lecture Notes in Economics and Mathematical Systems, vol. 374, pp. 119–134. Springer, Heidelberg (1992). Scholar
  15. 15.
    Klein, S., Staring, M., Murphy, K., Viergever, M.A., Pluim, J.P.W.: Elastix: a toolbox for intensity-based medical image registration. IEEE Trans. Med. Imaging 29(1), 196–205 (2010)CrossRefGoogle Scholar
  16. 16.
    Castillo, R., Castillo, E., Guerra, R., Johnson, V., McPhail, T., Garg, A., Guerrero, T.: A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets. Phys. Med. Biol. 54(7), 1849–1870 (2009)CrossRefGoogle Scholar
  17. 17.
    Papież, B.W., Heinrich, M.P., Fehrenbach, J., Risser, L., Schnabel, J.A.: An implicit sliding-motion preserving regularisation via bilateral filtering for deformable image registration. Med. Image Anal. 18(8), 1299–1311 (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Wei Sun
    • 1
    Email author
  • Dirk H. J. Poot
    • 2
  • Xuan Yang
    • 4
  • Wiro J. Niessen
    • 2
    • 3
  • Stefan Klein
    • 1
  1. 1.Department of Neurology, Donders Institute for Brain, Cognition and Behaviour, Donders Center for Medical NeuroscienceRadboud University Medical CenterNijmegenThe Netherlands
  2. 2.Biomedical Imaging Group RotterdamErasmus MCRotterdamThe Netherlands
  3. 3.Department of Image Science and Technology, Faculty of Applied SciencesDelft University of TechnologyDelftThe Netherlands
  4. 4.College of Computer Science and Software EngineeringShenzhen UniversityShenzhenChina

Personalised recommendations