Abstract
Investigations of biological ultrastructure, such as comprehensive mapping of connections within a nervous system, increasingly rely on large, high-resolution electron microscopy (EM) image volumes. However, discontinuities between the registered section images from which these volumes are assembled, due to variations in imaging conditions and section thickness, among other artifacts, impede truly 3-D analysis of these volumes. We propose an optimization procedure, called EMISAC (EM Image Stack Artifact Correction), to correct these discontinuities. EMISAC optimizes the parameters of spatially varying linear transformations of the data in order to minimize the squared norm of the gradient along the section axis, subject to detail-preserving regularization.
Assessment on a mouse cortex dataset demonstrates the effectiveness of our approach. Relative to the original data, EMISAC produces a large improvement both in NIQE score, a measure of statistical similarity between orthogonal cross-sections and the original image sections, as well as in accuracy of neurite segmentation, a critical task for this type of data. Compared to a recent independently-developed gradient-domain algorithm, EMISAC achieves significantly better NIQE image quality scores, and equivalent segmentation accuracy; future segmentation algorithms may be able to take advantage of the higher image quality.
In addition, on several time-lapse videos, EMISAC significantly reduces lighting artifacts, resulting in greatly improved video quality.
A software release is available at http://rll.berkeley.edu/2014_ ECCV_EMISAC .
This material is based upon work supported by the National Science Foundation under Grant No. 1118055.
Chapter PDF
Similar content being viewed by others
References
Alexander, A.L., Tsuruda, J.S., Parker, D.L.: Elimination of eddy current artifacts in diffusion-weighted echo-planar images: the use of bipolar gradients. Magnetic Resonance in Medicine 38(6), 1016–1021 (1997)
Andersson, J.L., Skare, S., Ashburner, J.: How to correct susceptibility distortions in spin-echo echo-planar images: application to diffusion tensor imaging. Neuroimage 20(2), 870–888 (2003)
Andres, B., Kroeger, T., Briggman, K.L., Denk, W., Korogod, N., Knott, G., Koethe, U., Hamprecht, F.A.: Globally optimal closed-surface segmentation for connectomics. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part III. LNCS, vol. 7574, pp. 778–791. Springer, Heidelberg (2012)
Andres, B., Köthe, U., Helmstaedter, M., Denk, W., Hamprecht, F.A.: Segmentation of SBFSEM volume data of neural tissue by hierarchical classification. In: Rigoll, G. (ed.) DAGM 2008. LNCS, vol. 5096, pp. 142–152. Springer, Heidelberg (2008)
Berger, D.R., Schalek, R., Kasthuri, N., Tapia, J.C., Hayworth, K., Seung, H.S., Lichtman, J.W.: SNEMI3D challenge, http://brainiac2.mit.edu/SNEMI3D/home (training volume)
Briggman, K.L., Bock, D.D.: Volume electron microscopy for neuronal circuit reconstruction. Current Opinion in Neurobiology 22(1), 154–161 (2012)
Cardona, A., Saalfeld, S., Preibisch, S., Schmid, B., Cheng, A., Pulokas, J., Tomancak, P., Hartenstein, V.: An integrated micro-and macroarchitectural analysis of the Drosophila brain by computer-assisted serial section electron microscopy. PLoS Biology 8(10), e1000502 (2010)
Cardona, A., Saalfeld, S., Preibisch, S., Schmid, B., Cheng, A., Pulokas, J., Tomancak, P., Hartenstein, V.: Segmented serial section Transmission Electron Microscopy (ssTEM) data set of the Drosophila first instar larva ventral nerve cord (VNC) (2010), http://www.ini.uzh.ch/~acardona/data.html
Chylinski, R.: Time-Lapse Photography: A Complete Guide to Shooting, Processing and Rendering Time-Lapse Movies. Cedar Wings Creative (2012), http://books.google.com/books?id=7fDaLPhJB5IC
Ciresan, D.C., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Deep neural networks segment neuronal membranes in electron microscopy images. In: NIPS, pp. 2852–2860 (2012)
Craddock, R.C., Jbabdi, S., Yan, C.G., Vogelstein, J.T., Castellanos, F.X., Di Martino, A., Kelly, C., Heberlein, K., Colcombe, S., Milham, M.P.: Imaging human connectomes at the macroscale. Nature Methods 10(6), 524–539 (2013)
Denk, W., Horstmann, H.: Serial block-face scanning electron microscopy to reconstruct three-dimensional tissue nanostructure. PLoS Biology 2(11), e329 (2004)
Haselgrove, J.C., Moore, J.R.: Correction for distortion of echo-planar images used to calculate the apparent diffusion coefficient. Magnetic Resonance in Medicine 36(6), 960–964 (1996)
Hayworth, K., Kasthuri, N., Schalek, R., Lichtman, J.: Automating the collection of ultrathin serial sections for large volume tem reconstructions. Microscopy and Microanalysis 12(S02), 86–87 (2006)
Helmstaedter, M.: Cellular-resolution connectomics: challenges of dense neural circuit reconstruction. Nature Methods 10(6), 501–507 (2013)
Helmstaedter, M., Briggman, K.L., Denk, W.: High-accuracy neurite reconstruction for high-throughput neuroanatomy. Nature Neuroscience 14(8), 1081–1088 (2011)
Jain, V., Bollmann, B., Richardson, M., Berger, D.R., Helmstaedter, M.N., Briggman, K.L., Denk, W., Bowden, J.B., Mendenhall, J.M., Abraham, W.C., et al.: Boundary learning by optimization with topological constraints. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2488–2495. IEEE (2010)
Jain, V., Murray, J.F., Roth, F., Turaga, S., Zhigulin, V., Briggman, K.L., Helmstaedter, M.N., Denk, W., Seung, H.S.: Supervised learning of image restoration with convolutional networks. In: IEEE International Conference on Computer Vision, pp. 1–8 (2007)
Jain, V., Turaga, S.C., Briggman, K.L., Helmstaedter, M.N., Denk, W., Seung, H.S.: Learning to agglomerate superpixel hierarchies. Advances in Neural Information Processing Systems 2(5) (2011)
Kasthuri, N., Lichtman, J.W.: Mouse S1 cortex Automatic Tape-Collecting Ultra Microtome (ATUM)-based Scanning Electron Microscopy (SEM) volume (2011), http://www.openconnectomeproject.org
Kaynig, V., Fischer, B., Müller, E., Buhmann, J.M.: Fully automatic stitching and distortion correction of transmission electron microscope images. Journal of Structural Biology 171(2), 163–173 (2010)
Kazhdan, M., Burns, R., Kasthuri, B., Lichtman, J., Vogelstein, J., Vogelstein, J.: Color corrected mouse S1 cortex Automatic Tape-Collecting Ultra Microtome (ATUM)-based Scanning Electron Microscopy (SEM) volume (2013), http://www.openconnectomeproject.org
Kazhdan, M., Burns, R., Kasthuri, B., Lichtman, J., Vogelstein, J., Vogelstein, J.: Gradient-domain processing for large em image stacks. arXiv preprint arXiv:1310.0041 (2013)
Knott, G., Marchman, H., Wall, D., Lich, B.: Serial section scanning electron microscopy of adult brain tissue using focused ion beam milling. The Journal of Neuroscience 28(12), 2959–2964 (2008)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1106–1114 (2012)
Kuwajima, M., Mendenhall, J.M., Harris, K.M.: Large-volume reconstruction of brain tissue from high-resolution serial section images acquired by sem-based scanning transmission electron microscopy. In: Nanoimaging, pp. 253–273. Springer (2013)
Meilă, M.: Comparing clusterings–an information based distance. Journal of Multivariate Analysis 98(5), 873–895 (2007)
Mittal, A., Soundararajan, R., Bovik, A.C.: Making a completely blind image quality analyzer. IEEE Signal Processing Letters 20(3), 209–212 (2013)
Nunez-Iglesias, J., Kennedy, R., Parag, T., Shi, J., Chklovskii, D.B.: Machine learning of hierarchical clustering to segment 2d and 3d images. PloS One 8(8), e71715 (2013)
Sage, D.: Local normalization filter to reduce the effect of non-uniform illumination (March 2011), http://bigwww.epfl.ch/sage/soft/localnormalization/
Schalek, R., Wilson, A., Lichtman, J., Josh, M., Kasthuri, N., Berger, D., Seung, S., Anger, P., Hayworth, K., Aderhold, D.: Atum-based sem for high-speed large-volume biological reconstructions. Microscopy and Microanalysis 18(S2), 572–573 (2012)
Turaga, S., Briggman, K., Helmstaedter, M., Denk, W., Seung, S.: Maximin affinity learning of image segmentation. In: Bengio, Y., Schuurmans, D., Lafferty, J., Williams, C.K.I., Culotta, A. (eds.) Advances in Neural Information Processing Systems, vol. 22, pp. 1865–1873. MIT Press, Cambridge (2009)
Turaga, S.C., Murray, J.F., Jain, V., Roth, F., Helmstaedter, M., Briggman, K., Denk, W., Seung, H.S.: Convolutional networks can learn to generate affinity graphs for image segmentation. Neural Comput. 22(2), 511–538 (2010)
Vazquez-Reina, A., Gelbart, M., Huang, D., Lichtman, J., Miller, E., Pfister, H.: Segmentation fusion for connectomics. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 177–184. IEEE (2011)
Wager, S., Wang, S., Liang, P.: Dropout training as adaptive regularization. In: Advances in Neural Information Processing Systems, pp. 351–359 (2013)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13(4), 600–612 (2004)
Zhu, C., Byrd, R.H., Lu, P., Nocedal, J.: Algorithm 778: L-bfgs-b: Fortran subroutines for large-scale bound-constrained optimization. ACM Transactions on Mathematical Software (TOMS) 23(4), 550–560 (1997)
Zuiderveld, K.: Contrast limited adaptive histograph equalization. Graphic Gems, pp. 474–485 (1994)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Azadi, S., Maitin-Shepard, J., Abbeel, P. (2014). Optimization-Based Artifact Correction for Electron Microscopy Image Stacks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds) Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol 8690. Springer, Cham. https://doi.org/10.1007/978-3-319-10605-2_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-10605-2_15
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-10604-5
Online ISBN: 978-3-319-10605-2
eBook Packages: Computer ScienceComputer Science (R0)