Deep Learning for Isotropic Super-Resolution from Non-isotropic 3D Electron Microscopy

  • Larissa Heinrich
  • John A. Bogovic
  • Stephan SaalfeldEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10434)


The most sophisticated existing methods to generate 3D isotropic super-resolution (SR) from non-isotropic electron microscopy (EM) are based on learned dictionaries. Unfortunately, none of the existing methods generate practically satisfying results. For 2D natural images, recently developed super-resolution methods that use deep learning have been shown to significantly outperform the previous state of the art. We have adapted one of the most successful architectures (FSRCNN) for 3D super-resolution, and compared its performance to a 3D U-Net architecture that has not been used previously to generate super-resolution. We trained both architectures on artificially downscaled isotropic ground truth from focused ion beam milling scanning EM (FIB-SEM) and tested the performance for various hyperparameter settings.

Our results indicate that both architectures can successfully generate 3D isotropic super-resolution from non-isotropic EM, with the U-Net performing consistently better. We propose several promising directions for practical application.


  1. 1.
    Abadi, M., Barham, P., Chen, J., Chen, Z., et al.: Tensorflow: a system for large-scale machine learning. In: 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2016), pp. 265–283 (2016)Google Scholar
  2. 2.
    Chollet, F.: Keras (2015).
  3. 3.
    Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199. Springer, Cham (2014). doi: 10.1007/978-3-319-10593-2_13CrossRefGoogle Scholar
  4. 4.
    Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 391–407. Springer, Cham (2016). doi: 10.1007/978-3-319-46475-6_25CrossRefGoogle Scholar
  5. 5.
    Glasner, D., Bagon, S., Irani, M.: Super-resolution from a single image. In: ICCV, pp. 349–356 (2009)Google Scholar
  6. 6.
    Glasner, D., et al.: High resolution segmentation of neuronal tissues from low depth-resolution EM imagery. In: Boykov, Y., Kahl, F., Lempitsky, V., Schmidt, F.R. (eds.) EMMCVPR 2011. LNCS, vol. 6819, pp. 261–272. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-23094-3_19CrossRefGoogle Scholar
  7. 7.
    Hanslovsky, P., Bogovic, J.A., Saalfeld, S.: Image-based correction of continuous and discontinuous non-planar axial distortion in serial section microscopy. Bioinformatics 33(9), 1379–1386 (2017). Scholar
  8. 8.
    Hayworth, K.J., Xu, C.S., Lu, Z., Knott, G.W., et al.: Ultrastructurally smooth thick partitioning and volume stitching for large-scale connectomics. Nat. Methods 12(4), 319–322 (2015)CrossRefGoogle Scholar
  9. 9.
    Hu, T., Nunez-Iglesias, J., Vitaladevuni, S., Scheffer, L., et al.: Super-resolution using sparse representations over learned dictionaries: reconstruction of brain structure using electron microscopy. arXiv preprint arXiv:1210.0564 (2012)
  10. 10.
    Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). doi: 10.1007/978-3-319-46475-6_43CrossRefGoogle Scholar
  11. 11.
    Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (2015)Google Scholar
  12. 12.
    Ledig, C., Theis, L., Huszar, F., Caballero, J., et al.: Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. arXiv:1609.04802 (2016)
  13. 13.
    Mikula, S.: Progress towards mammalian whole-brain cellular connectomics. Front. Neuroanat. 10, 62 (2016)CrossRefGoogle Scholar
  14. 14.
    Plaza, S.M., Scheffer, L.K., Chklovskii, D.B.: Toward large-scale connectome reconstructions. Curr. Opin. Neurobiol. 25, 201–210 (2014)CrossRefGoogle Scholar
  15. 15.
    Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). doi: 10.1007/978-3-319-24574-4_28CrossRefGoogle Scholar
  16. 16.
    Veeraraghavan, A., Genkin, A.V., Vitaladevuni, S., Scheffer, L., et al.: Increasing depth resolution of electron microscopy of neural circuits using sparse tomographic reconstruction. In: CVPR, pp. 1767–1774 (2010)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Larissa Heinrich
    • 1
  • John A. Bogovic
    • 1
  • Stephan Saalfeld
    • 1
    Email author
  1. 1.HHMI Janelia Research CampusAshburnUSA

Personalised recommendations