Image Models (and their Speech Model Cousins) pp 183-198 | Cite as

# Applications of the EM Algorithm to Linear Inverse Problems with Positivity Constraints

## Abstract

*Linear inverse* problems with *pos* itivity constraints (LININPOS problems, for short) are common in science and technology. Examples include deconvolution problems, signal recovery from linearly-distorting input-output systems, and more. In the absence of statistical noise, such problems are purely deterministic problems. It is therefore surprising that the EM algorithm (Baum et al., 1970, Dempster et al., 1977, and others) which was specifically developed as a tool for deriving maximum likelihood estimates (MLE’s) for complicated statistical models involving incomplete data, is also a useful tool for attacking LININPOS problems. This paper reviews the EM algorithm as a methodology for solving LININPOS problems and demonstrates it on two very different applications. The first is in the area of image analysis (specifically, “motion deblurring”) and the second is in the area of statistical inference from networks (specifically, ‘network tomography’).

### Keywords

Entropy Autocorrelation Convolution Deblurring## Preview

Unable to display preview. Download preview PDF.

### References

- Archer, G. E. B. and Titterington, D. M. (1993) On the iterative image space restoration algorithm for solving positive linear inverse problems. Preprint.Google Scholar
- Baum, L. E., Petrie, T., Soules, G. and Weiss, N. (1970) A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains.
*Ann. Math. Statist.*,**41**, 144–171.MathSciNetCrossRefGoogle Scholar - Byrne, C. L. (1993) Iterative image reconstruction algorithms based on cross-entropy minimization.
*IEEE Trans. on Image Processing*,**2**, 96–103.CrossRefGoogle Scholar - Csiszár, I. (1991) Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems.
*The Annals of Statistics*,**19**, No. 4, 2032–2066.MathSciNetMATHCrossRefGoogle Scholar - Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977) Maximum likelihood from incomplete data via the EM algorithm (with discussion).
*J. R. Statist. Soc. B*,**39**, 1–38.MathSciNetMATHGoogle Scholar - Eggermont, P. P. B. (1992) Nonlinear smoothing and the EM algorithm for positive integral equations of the first kind.
*Technical Report*. Department of Mathematics, University of Delaware, Newark.Google Scholar - Herman, G. T., Censor, Y. Gordon, D. and Lewitt, R. M. (1985) Comment on: A statistical model for PET by Vardi, Shepp and Kaufman,
*J. Am. Statist. Assoc.*,**80**, 22–25.CrossRefGoogle Scholar - Iusem, A. N. and Svaiter, B. F. (1994) A new regularization approach for a maximum likelihood estimation problem.
*App. Math. Optim.*,**29**, 225–241,MathSciNetMATHCrossRefGoogle Scholar - Lee, D. and Vardi, Y. (1994) Experiments with the EM/ML method on image motion deblurring. In
*Statistics and Images*(ed. K. V. Mardia), vol. II. Abingdon: Carfax.Google Scholar - Lucy, L. (1974) An iterative technique for the rectification of observed distributions,
*Astron. J.*,**79**, 745–754.CrossRefGoogle Scholar - Richardson, W. H. (1972) Bayesian-based iterative method of image restoration,
*J. Opt. Soc. Am.*,**62**, 55–59.CrossRefGoogle Scholar - Schulz, T. J. and Snyder, D. L. (1991) Imaging a randomly moving object from quantum-limited data: applications to image recovery from second- and third-order autocorrelations,
*J. Op. Soc. of Am. A*, 801–807.Google Scholar - Schulz, T. J. and Snyder, D. L. (1992) Image recovery from correlations,
*J. Op. Soc. of Am. A*, 1266–1272.Google Scholar - Shepp, L. A. and Vardi, Y. (1982) Maximum-likelihood reconstruction for emission tomography,
*IEEE Trans. Med. Imaging*,**MI-1**, 113–121.CrossRefGoogle Scholar - Snyder, D. L., Hammoud, A. M. and White, R. L. (1993) Image recovery from data acquired with a charge-coupled-device camera,
*J. Op. Soc of Am, A*, 1014–1023.Google Scholar - Snyder, D. L. and Schulz, T. J. (1990) High-resolution imaging at low-light levels through weak turbulence,
*J. Op. Soc of Am. A*, 1251–1265.Google Scholar - Snyder, D. L., Schulz, T. J. and O’Sullivan, J. A. (1992) De-blurring subject to nonnegativity constraints,
*IEEE Trans. Signal Processing*, 1143–1150.Google Scholar - Titterington, D. M. and Rossi, C (1985) Another look at a Bayesian direct deconvolution method.
*Signal Processing*, 9, 101–106.CrossRefGoogle Scholar - Vardi, Y. (1995) Network Tomography: Estimating source-destination traffic intensities from link data (fixed routing).
*J. Am. Statist. Assoc.*, (March 1996),**91**443, 365–377.MathSciNetGoogle Scholar - Vardi, Y. and Lee, D. (1993) From image deblurring to optimal investments: Maximum likelihood solution for positive linear inverse problems.
*J. R. Statist. Soc. B*,**4**(with discussion).Google Scholar - Vardi, Y., Shepp, L. A. and Kaufman, L. (1985) A statistical model for positron emission tomography,
*J. Am. Statist, Assoc.*, 80, 8–35.MathSciNetMATHCrossRefGoogle Scholar