Optimization-Based Artifact Correction for Electron Microscopy Image Stacks

  • Samaneh Azadi
  • Jeremy Maitin-Shepard
  • Pieter Abbeel
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8690)


Investigations of biological ultrastructure, such as comprehensive mapping of connections within a nervous system, increasingly rely on large, high-resolution electron microscopy (EM) image volumes. However, discontinuities between the registered section images from which these volumes are assembled, due to variations in imaging conditions and section thickness, among other artifacts, impede truly 3-D analysis of these volumes. We propose an optimization procedure, called EMISAC (EM Image Stack Artifact Correction), to correct these discontinuities. EMISAC optimizes the parameters of spatially varying linear transformations of the data in order to minimize the squared norm of the gradient along the section axis, subject to detail-preserving regularization.

Assessment on a mouse cortex dataset demonstrates the effectiveness of our approach. Relative to the original data, EMISAC produces a large improvement both in NIQE score, a measure of statistical similarity between orthogonal cross-sections and the original image sections, as well as in accuracy of neurite segmentation, a critical task for this type of data. Compared to a recent independently-developed gradient-domain algorithm, EMISAC achieves significantly better NIQE image quality scores, and equivalent segmentation accuracy; future segmentation algorithms may be able to take advantage of the higher image quality.

In addition, on several time-lapse videos, EMISAC significantly reduces lighting artifacts, resulting in greatly improved video quality.

A software release is available at ECCV_EMISAC .


Denoising Volume Electron Microscopy Connectomics Optimization Segmentation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alexander, A.L., Tsuruda, J.S., Parker, D.L.: Elimination of eddy current artifacts in diffusion-weighted echo-planar images: the use of bipolar gradients. Magnetic Resonance in Medicine 38(6), 1016–1021 (1997)CrossRefGoogle Scholar
  2. 2.
    Andersson, J.L., Skare, S., Ashburner, J.: How to correct susceptibility distortions in spin-echo echo-planar images: application to diffusion tensor imaging. Neuroimage 20(2), 870–888 (2003)CrossRefGoogle Scholar
  3. 3.
    Andres, B., Kroeger, T., Briggman, K.L., Denk, W., Korogod, N., Knott, G., Koethe, U., Hamprecht, F.A.: Globally optimal closed-surface segmentation for connectomics. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part III. LNCS, vol. 7574, pp. 778–791. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  4. 4.
    Andres, B., Köthe, U., Helmstaedter, M., Denk, W., Hamprecht, F.A.: Segmentation of SBFSEM volume data of neural tissue by hierarchical classification. In: Rigoll, G. (ed.) DAGM 2008. LNCS, vol. 5096, pp. 142–152. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  5. 5.
    Berger, D.R., Schalek, R., Kasthuri, N., Tapia, J.C., Hayworth, K., Seung, H.S., Lichtman, J.W.: SNEMI3D challenge, (training volume)
  6. 6.
    Briggman, K.L., Bock, D.D.: Volume electron microscopy for neuronal circuit reconstruction. Current Opinion in Neurobiology 22(1), 154–161 (2012)CrossRefGoogle Scholar
  7. 7.
    Cardona, A., Saalfeld, S., Preibisch, S., Schmid, B., Cheng, A., Pulokas, J., Tomancak, P., Hartenstein, V.: An integrated micro-and macroarchitectural analysis of the Drosophila brain by computer-assisted serial section electron microscopy. PLoS Biology 8(10), e1000502 (2010)Google Scholar
  8. 8.
    Cardona, A., Saalfeld, S., Preibisch, S., Schmid, B., Cheng, A., Pulokas, J., Tomancak, P., Hartenstein, V.: Segmented serial section Transmission Electron Microscopy (ssTEM) data set of the Drosophila first instar larva ventral nerve cord (VNC) (2010),
  9. 9.
    Chylinski, R.: Time-Lapse Photography: A Complete Guide to Shooting, Processing and Rendering Time-Lapse Movies. Cedar Wings Creative (2012),
  10. 10.
    Ciresan, D.C., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Deep neural networks segment neuronal membranes in electron microscopy images. In: NIPS, pp. 2852–2860 (2012)Google Scholar
  11. 11.
    Craddock, R.C., Jbabdi, S., Yan, C.G., Vogelstein, J.T., Castellanos, F.X., Di Martino, A., Kelly, C., Heberlein, K., Colcombe, S., Milham, M.P.: Imaging human connectomes at the macroscale. Nature Methods 10(6), 524–539 (2013)CrossRefGoogle Scholar
  12. 12.
    Denk, W., Horstmann, H.: Serial block-face scanning electron microscopy to reconstruct three-dimensional tissue nanostructure. PLoS Biology 2(11), e329 (2004)Google Scholar
  13. 13.
    Haselgrove, J.C., Moore, J.R.: Correction for distortion of echo-planar images used to calculate the apparent diffusion coefficient. Magnetic Resonance in Medicine 36(6), 960–964 (1996)CrossRefGoogle Scholar
  14. 14.
    Hayworth, K., Kasthuri, N., Schalek, R., Lichtman, J.: Automating the collection of ultrathin serial sections for large volume tem reconstructions. Microscopy and Microanalysis 12(S02), 86–87 (2006)CrossRefGoogle Scholar
  15. 15.
    Helmstaedter, M.: Cellular-resolution connectomics: challenges of dense neural circuit reconstruction. Nature Methods 10(6), 501–507 (2013)CrossRefGoogle Scholar
  16. 16.
    Helmstaedter, M., Briggman, K.L., Denk, W.: High-accuracy neurite reconstruction for high-throughput neuroanatomy. Nature Neuroscience 14(8), 1081–1088 (2011)CrossRefGoogle Scholar
  17. 17.
    Jain, V., Bollmann, B., Richardson, M., Berger, D.R., Helmstaedter, M.N., Briggman, K.L., Denk, W., Bowden, J.B., Mendenhall, J.M., Abraham, W.C., et al.: Boundary learning by optimization with topological constraints. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2488–2495. IEEE (2010)Google Scholar
  18. 18.
    Jain, V., Murray, J.F., Roth, F., Turaga, S., Zhigulin, V., Briggman, K.L., Helmstaedter, M.N., Denk, W., Seung, H.S.: Supervised learning of image restoration with convolutional networks. In: IEEE International Conference on Computer Vision, pp. 1–8 (2007)Google Scholar
  19. 19.
    Jain, V., Turaga, S.C., Briggman, K.L., Helmstaedter, M.N., Denk, W., Seung, H.S.: Learning to agglomerate superpixel hierarchies. Advances in Neural Information Processing Systems 2(5) (2011)Google Scholar
  20. 20.
    Kasthuri, N., Lichtman, J.W.: Mouse S1 cortex Automatic Tape-Collecting Ultra Microtome (ATUM)-based Scanning Electron Microscopy (SEM) volume (2011),
  21. 21.
    Kaynig, V., Fischer, B., Müller, E., Buhmann, J.M.: Fully automatic stitching and distortion correction of transmission electron microscope images. Journal of Structural Biology 171(2), 163–173 (2010)CrossRefGoogle Scholar
  22. 22.
    Kazhdan, M., Burns, R., Kasthuri, B., Lichtman, J., Vogelstein, J., Vogelstein, J.: Color corrected mouse S1 cortex Automatic Tape-Collecting Ultra Microtome (ATUM)-based Scanning Electron Microscopy (SEM) volume (2013),
  23. 23.
    Kazhdan, M., Burns, R., Kasthuri, B., Lichtman, J., Vogelstein, J., Vogelstein, J.: Gradient-domain processing for large em image stacks. arXiv preprint arXiv:1310.0041 (2013)Google Scholar
  24. 24.
    Knott, G., Marchman, H., Wall, D., Lich, B.: Serial section scanning electron microscopy of adult brain tissue using focused ion beam milling. The Journal of Neuroscience 28(12), 2959–2964 (2008)CrossRefGoogle Scholar
  25. 25.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1106–1114 (2012)Google Scholar
  26. 26.
    Kuwajima, M., Mendenhall, J.M., Harris, K.M.: Large-volume reconstruction of brain tissue from high-resolution serial section images acquired by sem-based scanning transmission electron microscopy. In: Nanoimaging, pp. 253–273. Springer (2013)Google Scholar
  27. 27.
    Meilă, M.: Comparing clusterings–an information based distance. Journal of Multivariate Analysis 98(5), 873–895 (2007)CrossRefzbMATHMathSciNetGoogle Scholar
  28. 28.
    Mittal, A., Soundararajan, R., Bovik, A.C.: Making a completely blind image quality analyzer. IEEE Signal Processing Letters 20(3), 209–212 (2013)CrossRefGoogle Scholar
  29. 29.
    Nunez-Iglesias, J., Kennedy, R., Parag, T., Shi, J., Chklovskii, D.B.: Machine learning of hierarchical clustering to segment 2d and 3d images. PloS One 8(8), e71715 (2013)Google Scholar
  30. 30.
    Sage, D.: Local normalization filter to reduce the effect of non-uniform illumination (March 2011),
  31. 31.
    Schalek, R., Wilson, A., Lichtman, J., Josh, M., Kasthuri, N., Berger, D., Seung, S., Anger, P., Hayworth, K., Aderhold, D.: Atum-based sem for high-speed large-volume biological reconstructions. Microscopy and Microanalysis 18(S2), 572–573 (2012)Google Scholar
  32. 32.
    Turaga, S., Briggman, K., Helmstaedter, M., Denk, W., Seung, S.: Maximin affinity learning of image segmentation. In: Bengio, Y., Schuurmans, D., Lafferty, J., Williams, C.K.I., Culotta, A. (eds.) Advances in Neural Information Processing Systems, vol. 22, pp. 1865–1873. MIT Press, Cambridge (2009)Google Scholar
  33. 33.
    Turaga, S.C., Murray, J.F., Jain, V., Roth, F., Helmstaedter, M., Briggman, K., Denk, W., Seung, H.S.: Convolutional networks can learn to generate affinity graphs for image segmentation. Neural Comput. 22(2), 511–538 (2010)CrossRefzbMATHGoogle Scholar
  34. 34.
    Vazquez-Reina, A., Gelbart, M., Huang, D., Lichtman, J., Miller, E., Pfister, H.: Segmentation fusion for connectomics. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 177–184. IEEE (2011)Google Scholar
  35. 35.
    Wager, S., Wang, S., Liang, P.: Dropout training as adaptive regularization. In: Advances in Neural Information Processing Systems, pp. 351–359 (2013)Google Scholar
  36. 36.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13(4), 600–612 (2004)CrossRefGoogle Scholar
  37. 37.
    Zhu, C., Byrd, R.H., Lu, P., Nocedal, J.: Algorithm 778: L-bfgs-b: Fortran subroutines for large-scale bound-constrained optimization. ACM Transactions on Mathematical Software (TOMS) 23(4), 550–560 (1997)CrossRefzbMATHMathSciNetGoogle Scholar
  38. 38.
    Zuiderveld, K.: Contrast limited adaptive histograph equalization. Graphic Gems, pp. 474–485 (1994)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Samaneh Azadi
    • 1
  • Jeremy Maitin-Shepard
    • 1
  • Pieter Abbeel
    • 1
  1. 1.EECS DepartmentUniversity of California at BerkeleyUSA

Personalised recommendations