Skip to main content
SpringerLink
Log in

EM Algorithms from a Non-stochastic Perspective

  • Reference work entry
Handbook of Mathematical Methods in Imaging

Abstract

The EM algorithm is not a single algorithm, but a template for the construction of iterative algorithms. While it is always presented in stochastic language, relying on conditional expectations to obtain a method for estimating parameters in statistics, the essence of the EM algorithm is not stochastic. The conventional formulation of the EM algorithm given in many texts and papers on the subject is inadequate. A new formulation is given here based on the notion of acceptable data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 1,200.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 549.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. McLachlan, G., Krishnan, T.: The EM Algorithm and Extensions. Wiley, New York (1997)

    MATH  Google Scholar 

  2. Meng, X., Pedlow, S.: EM: a bibliographic review with missing articles. In: Proceedings of the Statistical Computing Section, American Statistical Association. American Statistical Association, Alexandria (1992)

    Google Scholar 

  3. Meng, X., van Dyk, D.: The EM algorithm- an old folk-song sung to a fast new tune. J. R. Stat. Soc. B 59(3), 511–567 (1997)

    Article  MATH  Google Scholar 

  4. Becker, M., Yang, I., Lange, K.: EM algorithms without missing data. Stat. Methods Med. Res. 6, 38–54 (1997)

    Article  Google Scholar 

  5. Byrne, C.: Iterative Optimization in Inverse Problems. Taylor and Francis, Boca Raton (2014)

    MATH  Google Scholar 

  6. Byrne, C.: Non-stochastic EM algorithms in optimization. J. Nonlinear Convex Anal. (to appear, 2015)

    Google Scholar 

  7. Kullback, S., Leibler, R.: On information and sufficiency. Ann. Math. Stat. 22, 79–86 (1951)

    Article  MATH  MathSciNet  Google Scholar 

  8. Hogg, R., McKean, J., Craig, A.: Introduction to Mathematical Statistics, 6th edn. Prentice Hall, Englewood Cliffs (2004)

    Google Scholar 

  9. Byrne, C., Eggermont, P.: EM algorithms. In: Scherzer, O. (ed.) Handbook of Mathematical Methods in Imaging. Springer, New York (2010)

    Google Scholar 

  10. Fessler, J., Ficaro, E., Clinthorne, N., Lange, K.: Grouped-coordinate ascent algorithms for penalized-likelihood transmission image reconstruction. IEEE Trans. Med. Imaging 16(2), 166–175 (1997)

    Article  Google Scholar 

  11. Dempster, A., Laird, N., Rubin, D.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. B 37, 1–38 (1977)

    MathSciNet  Google Scholar 

  12. Csiszár, I., Tusnády, G.: Information geometry and alternating minimization procedures. Stat. Decis. (Suppl. 1), R. Oldenbourg Verlag, München, 205–237 (1984)

    Google Scholar 

  13. Byrne, C.: Alternating and sequential unconstrained minimization algorithms. J. Optim. Theory Appl. Electron. 154(3), doi:10.1007/s1090134-2; hardcopy 156(2) (2012)

    Google Scholar 

  14. Eggermont, P., LaRiccia, V.: Smoothed maximum likelihood density estimation for inverse problems. Ann. Stat. 23, 199–220 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  15. Eggermont, P., LaRiccia, V.: Maximum Penalized Likelihood Estimation, Volume I: Density Estimation. Springer, New York (2001)

    Google Scholar 

  16. Bauschke, H., Combettes, P., Noll, D.: Joint minimization with alternating Bregman proximity operators. Pac. J. Opt. 2, 401–424 (2006)

    MATH  MathSciNet  Google Scholar 

  17. Wernick, M., Aarsvold, J. (eds.): Emission Tomography: The Fundamentals of PET and SPECT. Elsevier Academic, San Diego (2004)

    Google Scholar 

  18. Rockmore, A., Macovski, A.: A maximum likelihood approach to emission image reconstruction from projections. IEEE Trans. Nucl. Sc. NS-23, 1428–1432 (1976)

    Article  Google Scholar 

  19. Shepp, L., Vardi, Y.: Maximum likelihood reconstruction for emission tomography. IEEE Trans. Med. Imaging MI-1, 113–122 (1982)

    Article  Google Scholar 

  20. Byrne, C.: Iterative image reconstruction algorithms based on cross-entropy minimization. IEEE Trans. Image Proc. IP-2, 96–103 (1993)

    Article  Google Scholar 

  21. Byrne, C.: Block-iterative methods for image reconstruction from projections. IEEE Trans. Image Proc. IP-5, 792–794 (1996)

    Article  Google Scholar 

  22. Byrne, C.: Convergent block-iterative algorithms for image reconstruction from inconsistent data. IEEE Trans. Image Proc. IP-6, 1296–1304 (1997)

    Article  MathSciNet  Google Scholar 

  23. Byrne, C.: Accelerating the EMML algorithm and related iterative algorithms by rescaled block-iterative (RBI) methods. IEEE Trans. Image Proc. IP-7, 100–109 (1998)

    Article  MathSciNet  Google Scholar 

  24. Gordon, R., Bender, R., Herman, G.T.: Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and x-ray photography. J. Theor. Biol. 29, 471–481 (1970)

    Article  Google Scholar 

  25. Darroch, J., Ratcliff, D.: Generalized iterative scaling for log-linear models. Ann. Math. Stat. 43, 1470–1480 (1972)

    Article  MATH  MathSciNet  Google Scholar 

  26. Schmidlin, P.: Iterative separation of sections in tomographic scintigrams. Nucl. Med. 15(1), 1–16 (1972)

    Google Scholar 

  27. Censor, Y., Segman, J.: On block-iterative maximization. J. Inf. Opt. Sci. 8, 275–291 (1987)

    MATH  MathSciNet  Google Scholar 

  28. Byrne, C.: Iterative reconstruction algorithms based on cross-entropy minimization. In: Shepp, L., Levinson, S.E. (eds.) Image Models (and their Speech Model Cousins). IMA Volumes in Mathematics and its Applications vol. 80, pp. 1–11. Springer, New York (1996)

    Chapter  Google Scholar 

  29. Narayanan, M., Byrne, C., King, M.: An interior point iterative maximum-likelihood reconstruction algorithm incorporating upper and lower bounds with application to SPECT transmission imaging. IEEE Trans. Med. Imaging TMI-20(4), 342–353 (2001)

    Article  Google Scholar 

  30. Byrne, C.: Iterative algorithms for deblurring and deconvolution with constraints. Inverse Probl. 14, 1455–1467 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  31. Everitt, B., Hand, D.: Finite Mixture Distributions. Chapman and Hall, London (1981)

    Book  MATH  Google Scholar 

  32. Redner, R., Walker, H.: Mixture densities, maximum likelihood and the EM algorithm. SIAM Rev. 26(2), 195–239 (1984)

    Article  MATH  MathSciNet  Google Scholar 

  33. Byrne, C.: Likelihood maximization for list-mode emission tomographic image reconstruction. IEEE Trans. Med. Imaging 20(10), 1084–1092 (2001)

    Article  Google Scholar 

  34. Wu, C.F.J.: On the convergence properties of the EM algorithm. Ann. Stat. 11, 95–103 (1983)

    Article  MATH  Google Scholar 

  35. Boyles, R.: On the convergence of the EM algorithm. J. R. Stat. Soc. B 45, 47–50 (1983)

    MATH  MathSciNet  Google Scholar 

  36. Lange, K., Carson, R.: EM reconstruction algorithms for emission and transmission tomography. J. Comput. Assist. Tomogr. 8, 306–316 (1984)

    Google Scholar 

  37. Vardi, Y., Shepp, L.A., Kaufman, L.: A statistical model for positron emission tomography. J. Am. Stat. Assoc. 80, 8–20 (1985)

    Article  MATH  MathSciNet  Google Scholar 

  38. Lange, K., Bahn, M., Little, R.: A theoretical study of some maximum likelihood algorithms for emission and transmission tomography. IEEE Trans. Med. Imaging MI-6(2), 106–114 (1987)

    Article  Google Scholar 

  39. Barrett, H., White, T., Parra, L.: List-mode likelihood. J. Opt. Soc. Am. A 14, 2914–2923 (1997)

    Article  Google Scholar 

  40. Parra, L., Barrett, H.: List-mode likelihood: EM algorithm and image quality estimation demonstrated on 2-D PET. IEEE Trans. Med. Imaging 17, 228–235 (1998)

    Article  Google Scholar 

  41. Huesman, R., Klein, G., Moses, W., Qi, J., Ruetter, B., Virador, P.: List-mode maximum likelihood reconstruction applied to positron emission mammography (PEM) with irregular sampling. IEEE Trans. Med. Imaging 19(5), 532–537 (2000)

    Article  Google Scholar 

Download references

Acknowledgements

I wish to thank Professor Paul Eggermont of the University of Delaware for helpful discussions on these matters.

Author information

Authors and Affiliations

  1. Department of Mathematical Sciences, University of Massachusetts Lowell, 01854, Lowell, MA, USA

    Charles Byrne

Authors
  1. Charles Byrne
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Charles Byrne .

Editor information

Editors and Affiliations

  1. Computational Science Center, University of Vienna, Vienna, Austria

    Otmar Scherzer

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer Science+Business Media New York

About this entry

Cite this entry

Byrne, C. (2015). EM Algorithms from a Non-stochastic Perspective. In: Scherzer, O. (eds) Handbook of Mathematical Methods in Imaging. Springer, New York, NY. https://doi.org/10.1007/978-1-4939-0790-8_46

Download citation

Publish with us

Policies and ethics

Search

Navigation