Abstract
Nowadays, several medical procedures depend on the comparison and combination of images obtained in different modalities (magnetic resonance, computed tomography, PET, among others). Image registration is a geometric transformation process to align two or more images. It is necessary to have robust algorithms to find the best parameters of transformation in order to achieve accurate registrations. Reinforcement learning allows to train an agent through direct environment interaction, to achieve a goal. In this work, a comparison of the performance of Q-learning and Deep-Q with its variants is presented. Brain magnetic resonance images are used in 2D domain considering rigid deformations. The comparison is based on the reward values, computing the Pearson correlation factor in monomodal registration and Mutual information in multimodal registration, obtained during the learning process. It is also considered an error measure between the target parameters and the achieved ones. Finally, a backup memory criterion is proposed to train the Q-Network methods. Experimental results show a successfully behavior in all cases, but performance is improved when the proposed criterion is applied.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. (2017)
Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: a survey. J. Artif. Intell. Res. 4, 237–285 (1996)
Zitová, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21, 977–1000 (2003)
Maintz, J.B.A., Viergever, M.A.: A survey of medical image registration. Med. Image Anal. 2(1), 1–37 (1998)
Roderick, M., MacGlashan, J., Tellex, S.: Implementing the deep Q-network, November 2017
Sigaud, O., Buffet, O.: Markov decision processes in artificial intelligence (2010)
Watkins, C.J.C.H., Dayan, P.: Q-learning. Mach. Learn. 8(3–4), 279–292 (1992)
Bellman, R.: A Markovian decision process (1957)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double Q-learning, September 2015
Lin, L.-J.: Self-improving reactive agents based on reinforcement learning, planning and teaching. Mach. Learn. 8(3–4), 293–321 (1992)
Wang, Z., Schaul, T., Hessel, M., Com, M., Van Hasselt, H., Lanctot, M.: Dueling network architectures for deep reinforcement learning (2015)
Harmon, M.E., Baird, L.C., Klopr, A.H.: Advantage updating applied to a differential game. In: NIPS (1995)
Fitzpatrick, J.M., Hill, D.L.G., Maurer, C.R.: Image registration. In: Handbook of Medical Imaging. Medical Image Processing and Analysis, vol. 2, pp. 447–514 (2000)
Hoffer, E., Hubara, I., Soudry, D.: Train longer, generalize better: closing the generalization gap in large batch training of neural networks (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
The authors declare that they have no conflict of interest.
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Isa-Jara, R.F., Meschino, G.J., Ballarin, V.L. (2020). A Comparative Study of Reinforcement Learning Algorithms Applied to Medical Image Registration. In: González DÃaz, C., et al. VIII Latin American Conference on Biomedical Engineering and XLII National Conference on Biomedical Engineering. CLAIB 2019. IFMBE Proceedings, vol 75. Springer, Cham. https://doi.org/10.1007/978-3-030-30648-9_36
Download citation
DOI: https://doi.org/10.1007/978-3-030-30648-9_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30647-2
Online ISBN: 978-3-030-30648-9
eBook Packages: EngineeringEngineering (R0)