Abstract
In 3D medical imaging, images with isotropic high resolution (HR) are almost always preferred. In practice, however, many acquired images, including magnetic resonance imaging (MRI) and fluorescence microscopy, have HR in the in-plane directions and low resolution (LR) in the through-plane direction. The blurriness and aliasing artifacts that result cannot be solved by simple interpolation. Instead, many researchers have proposed super-resolution algorithms including state-of-art convolutional neural network (CNN)-based methods that require matched training data that have paired LR/HR examples. Since these data are often unavailable in practice, self super-resolution algorithms that do not need external training data have also been proposed. These self super-resolution methods assume that the in-plane slices are HR, and can therefore be used as HR training data. By degrading them into LR images, 2D CNNs can be trained and then used to restore the images in the through-plane. However, there are two issues with these approaches. The first one is that the assumption of HR in-plane slices is actually not solid since these thick in-plane slices are averaged true HR thin slices. Training on thick slices is equivalent to training on averaged true HR images, which is suboptimal. The second one relates to the 2D CNNs used on 3D volume, which cannot guarantee slice consistency. Regarding both issues as well as the generalizability of algorithm, we made four contributions. We show in this paper that one of the existing 2D CNN-based self super-resolution methods, SMORE, can be further improved by iteratively applying it using 2D or 3D networks, yielding 2D and 3D iSMORE. This iterative framework improves training data from thick slices to thinner slices after each iteration, thus improves super-resolution accuracy after each iteration, and solves the first issue. The second contribution is that it uses a 3D network to preserve slice consistency. The third contribution is the use of an edge-based loss function and noise reduction to enhance the performance. Finally, we perform iSMORE on both MRI and two-photon fluorescence microscopy, which demonstrates its generalizability.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bei, Y., Damian, A., Hu, S., Menon, S., Ravi, N., Rudin, C.: New techniques for preserving global structure and denoising with low information loss in single-image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 874–881 (2018)
Helmchen, F., Denk, W.: Deep tissue two-photon microscopy. Nat. Methods 2(12), 932 (2005)
Jog, A., Carass, A., Prince, J.L.: Self super-resolution for magnetic resonance images. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9902, pp. 553–560. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46726-9_64
Lim, B., Son, S., Kim, H., Nah, S., Mu Lee, K.: Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 136–144 (2017)
Ragan, T., et al.: Serial two-photon tomography for automated ex vivo mouse brain imaging. Nat. Methods 9(3), 255 (2012)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Timofte, R., De Smet, V., Van Gool, L.: Anchored neighborhood regression for fast example-based super-resolution. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1920–1927 (2013)
Timofte, R., Gu, S., Wu, J., Van Gool, L.: Ntire 2018 challenge on single image super-resolution: methods and results. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 852–863 (2018)
Weigert, M., et al.: Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat. Methods 15(12), 1090 (2018)
Zhao, C., Carass, A., Dewey, B.E., Prince, J.L.: Self super-resolution for magnetic resonance images using deep networks. In: IEEE International Symposium on Biomedical Imaging (ISBI) (2018)
Zhao, C., et al.: A deep learning based anti-aliasing self super-resolution algorithm for MRI. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 100–108. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_12
Zhao, C., et al.: Applications of a deep learning method for anti-aliasing and super-resolution in MRI. Magnetic Resonance Imaging (2019)
Acknowledgments
This work was supported by the NIH under grant R01-NS108407, R01-NS105503, and the National Multiple Sclerosis Society grant RG-1507-05243. Thanks to Aaron Carass for valuable discussion.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, C., Son, S., Kim, Y., Prince, J.L. (2019). iSMORE: An Iterative Self Super-Resolution Algorithm. In: Burgos, N., Gooya, A., Svoboda, D. (eds) Simulation and Synthesis in Medical Imaging. SASHIMI 2019. Lecture Notes in Computer Science(), vol 11827. Springer, Cham. https://doi.org/10.1007/978-3-030-32778-1_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-32778-1_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32777-4
Online ISBN: 978-3-030-32778-1
eBook Packages: Computer ScienceComputer Science (R0)