Applications of the EM Algorithm to Linear Inverse Problems with Positivity Constraints

  • Y. Vardi
Part of the The IMA Volumes in Mathematics and its Applications book series (IMA, volume 80)


Linear inverse problems with pos itivity constraints (LININPOS problems, for short) are common in science and technology. Examples include deconvolution problems, signal recovery from linearly-distorting input-output systems, and more. In the absence of statistical noise, such problems are purely deterministic problems. It is therefore surprising that the EM algorithm (Baum et al., 1970, Dempster et al., 1977, and others) which was specifically developed as a tool for deriving maximum likelihood estimates (MLE’s) for complicated statistical models involving incomplete data, is also a useful tool for attacking LININPOS problems. This paper reviews the EM algorithm as a methodology for solving LININPOS problems and demonstrates it on two very different applications. The first is in the area of image analysis (specifically, “motion deblurring”) and the second is in the area of statistical inference from networks (specifically, ‘network tomography’).


Point Spread Function Probability Vector Moment Equation Positivity Constraint Picture Frame 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Archer, G. E. B. and Titterington, D. M. (1993) On the iterative image space restoration algorithm for solving positive linear inverse problems. Preprint.Google Scholar
  2. Baum, L. E., Petrie, T., Soules, G. and Weiss, N. (1970) A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Ann. Math. Statist., 41, 144–171.MathSciNetCrossRefGoogle Scholar
  3. Byrne, C. L. (1993) Iterative image reconstruction algorithms based on cross-entropy minimization. IEEE Trans. on Image Processing, 2, 96–103.CrossRefGoogle Scholar
  4. Csiszár, I. (1991) Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems. The Annals of Statistics, 19, No. 4, 2032–2066.MathSciNetMATHCrossRefGoogle Scholar
  5. Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977) Maximum likelihood from incomplete data via the EM algorithm (with discussion). J. R. Statist. Soc. B, 39, 1–38.MathSciNetMATHGoogle Scholar
  6. Eggermont, P. P. B. (1992) Nonlinear smoothing and the EM algorithm for positive integral equations of the first kind. Technical Report. Department of Mathematics, University of Delaware, Newark.Google Scholar
  7. Herman, G. T., Censor, Y. Gordon, D. and Lewitt, R. M. (1985) Comment on: A statistical model for PET by Vardi, Shepp and Kaufman, J. Am. Statist. Assoc., 80, 22–25.CrossRefGoogle Scholar
  8. Iusem, A. N. and Svaiter, B. F. (1994) A new regularization approach for a maximum likelihood estimation problem. App. Math. Optim., 29, 225–241,MathSciNetMATHCrossRefGoogle Scholar
  9. Lee, D. and Vardi, Y. (1994) Experiments with the EM/ML method on image motion deblurring. In Statistics and Images (ed. K. V. Mardia), vol. II. Abingdon: Carfax.Google Scholar
  10. Lucy, L. (1974) An iterative technique for the rectification of observed distributions, Astron. J., 79, 745–754.CrossRefGoogle Scholar
  11. Richardson, W. H. (1972) Bayesian-based iterative method of image restoration, J. Opt. Soc. Am., 62, 55–59.CrossRefGoogle Scholar
  12. Schulz, T. J. and Snyder, D. L. (1991) Imaging a randomly moving object from quantum-limited data: applications to image recovery from second- and third-order autocorrelations, J. Op. Soc. of Am. A, 801–807.Google Scholar
  13. Schulz, T. J. and Snyder, D. L. (1992) Image recovery from correlations, J. Op. Soc. of Am. A, 1266–1272.Google Scholar
  14. Shepp, L. A. and Vardi, Y. (1982) Maximum-likelihood reconstruction for emission tomography, IEEE Trans. Med. Imaging, MI-1, 113–121.CrossRefGoogle Scholar
  15. Snyder, D. L., Hammoud, A. M. and White, R. L. (1993) Image recovery from data acquired with a charge-coupled-device camera, J. Op. Soc of Am, A, 1014–1023.Google Scholar
  16. Snyder, D. L. and Schulz, T. J. (1990) High-resolution imaging at low-light levels through weak turbulence, J. Op. Soc of Am. A, 1251–1265.Google Scholar
  17. Snyder, D. L., Schulz, T. J. and O’Sullivan, J. A. (1992) De-blurring subject to nonnegativity constraints, IEEE Trans. Signal Processing, 1143–1150.Google Scholar
  18. Titterington, D. M. and Rossi, C (1985) Another look at a Bayesian direct deconvolution method. Signal Processing, 9, 101–106.CrossRefGoogle Scholar
  19. Vardi, Y. (1995) Network Tomography: Estimating source-destination traffic intensities from link data (fixed routing). J. Am. Statist. Assoc., (March 1996), 91 443, 365–377.MathSciNetGoogle Scholar
  20. Vardi, Y. and Lee, D. (1993) From image deblurring to optimal investments: Maximum likelihood solution for positive linear inverse problems. J. R. Statist. Soc. B, 4 (with discussion).Google Scholar
  21. Vardi, Y., Shepp, L. A. and Kaufman, L. (1985) A statistical model for positron emission tomography, J. Am. Statist, Assoc., 80, 8–35.MathSciNetMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag New York, Inc. 1996

Authors and Affiliations

  • Y. Vardi
    • 1
    • 2
  1. 1.Rutgers UniversityNew BrunswickUSA
  2. 2.AT&T Bell LaboratoriesMurray HillUSA

Personalised recommendations