Abstract
We present a family of algorithms of the row-action type that are suitable for solving large and sparse entropy maximization problems with linear constraints. The algorithms are designed to handle equality constraints, in-equality constraints, or interval constraints (that is, pairs of inequalities). Six iterative algorithms are discussed, but for one of them convergence of the process is not yet proven. No experimental results are presented.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bregman, L. M. (1967a), “The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming,” USSR Computational Mathematics and Mathematical Phys. 7(3), pp. 200–217.
Bregman, L. M. (1967b), “Proof of convergence of Sheleikhovskii’s method for a problem with transportation constraints,” USSR Computational Math, and Mathemat. Phys. 7, pp. 191–204.
Censor, Y. (1981a), “Row-action methods for huge and sparse systems and their applications,” SIAM Review 23, pp. 444–466.
Censor, Y. (1981b), “Intervals in linear and nonlinear problems of image reconstruction,” in Mathematical Aspects of Computerized Tomography, G. T. Herman and F. Natterer, eds., Lecture Notes in Medical Informatics, Springer-Verlag, Berlin, Vol. 8, pp. 152–159.
Censor, Y. (1982), “Entropy optimization via entropy projections,” in System Modeling and Optimization, R. F. Drenick and F. Kozin, eds., Lecture Notes in Control and Information Science, Springer-Verlag, Berlin, Vol. 38, pp. 450–454.
Censor, Y. (1983), “Finite series expansion reconstruction methods,” Proc. IEEE 71, pp. 409–419.
Censor, Y., A. R. De Pierro, T. Elfving, G. T. Herman, and A. N. lusem (1986), “On iterative methods for linearly constrained entropy maximization,” Tech. Rept. MIPG112, Medical Image Processing Group, Department of Radiology, Hospital of the University of Pennsylvania, Philadelphia, Pa.
Censor, Y., A. R. De Pierro, and A. N. lusem (1986), “On maximization of entropies and a generalization of Bregman’s method for convex programming,” Tech. Rept. MIPG113, Medical Image Processing Group, Department of Radiology, Hospital of the University of Pennsylvania, Philadelphia, Pa.
Censor, Y., P. P. B. Eggermont, and D. Gordon (1983), “Strong underrelaxation in Kaczmarz’s method for inconsistent systems,” Numerische Mathematik 41, pp. 83–92.
Censor, Y., T. Elfving, and G. T. Herman (1983), “Methods for entropy maximization with applications in image processing,” in Proceedings of the Third Scandinavian Conference on Image Analysis, P. Johansen and P. W. Becker, eds., Chartwell-Bratt, Ltd., Lund, Sweden, pp. 296–300.
Censor, Y., A. V. Lakshminarayanan, and A. Lent (1979), “Relaxational methods for large-scale entropy optimization problems with application in image reconstruction,” in Information Linkage Between Applied Mathematics and Industry, P. C. C. Wang et al., eds., Academic Press, New York, pp. 539–546.
Censor, Y., and A. Lent (1981), “An iterative row-action method for interval convex programming,” J. Optimization Theory and Applications 34, pp. 321–353.
Censor, Y., and A. Lent (1981), “An iterative row-action method for interval convex programming,” J. Optimization Theory and Applications 34, pp. 321–353.
Darroch, J. N., and D. Ratcliff (1972), “Generalized iterative scaling for log-linear models,” Ann. Math. Statis. 43, pp. 1470–1480.
Eggermont, P. P. B., G. T. Herman, and A. Lent (1981), “Iterative algorithms for large partitioned linear systems with applications to image reconstruction,” Linear Algebra and Its Applications 40, pp. 37–67.
Elfving, T. (1980), “On some methods for entropy maximization and matrix scaling,” Linear Algebra and Its Applications 34, pp. 321–339.
Erlander, S. (1981), “Entropy in linear programs,” Mathematical Programming 21, pp. 137–151.
Frieden, B. R (1980), “Statistical models for the image restoration problem,” Computer Graphics and Image Processing 12, pp. 40–59,
Frieden, B. R. (1983), Probability, Statistical Optics, and Data Testing, Springer-Verlag, New York.
Gordon, R., R. Bender, and G. T. Herman (1970), “Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and x-ray photography,” J. Theoret. Biol. 29, pp. 471–481.
Herman, G. T. (1975), “A relaxation method for reconstructing objects from noisy x-rays,” Mathemat. Programming 8, pp. 1–19.
Herman, G. T. (1980), Image Reconstruction from Projections: The Fundamentals of Computerized Tomography, Academic Press, New York.
Herman, G. T. (1982a), “Mathematical optimization versus practical performance: a case study based on the maximum entropy criterion in image reconstruction,” Mathemat. Programming Study 20, pp. 96–112.
Herman, G. T. (1982b), “The application of maximum entropy and Bayesian optimization methods to image reconstruction from projections,” Tech. Rept. MIPG73, Medical Image Processing Group, Dept. Radiology, Hospital of the University of Pennsylvania, Philadelphia.
Herman, G. T., and A. Lent (1978), “A family of iterative quadratic optimization algorithms for pairs of inequalities, with application in diagnostic radiology,” Mathemat. Programming Study 9, pp. 15–29.
Hildreth, C. (1957), “A quadratic programming procedure,” Naval Res. Logistic Quart. 4, pp. 79–85; erratum, p. 361.
Jaynes, E. T. (1982), “On the rationale of maximum-entropy methods,” Proc. IEEE 70, pp. 939–952.
Kapur, J. N. (1983), “Twenty-five years of maximum-entropy principle,” J. Mathemat. Phys. Sci. 17, pp. 103–156.
Kruithof, J. (1937), “Telefoonverkeersrekening” (Calculation of telephone traffic), De Ingenieur 52, pp. E15–E25.
Krupp, R. S (1979), “Properties of Kruithof’s projection method,” Bell Syst. Tech. J. 58, pp. 517–538.
Lamond, B., and N. F. Stewart (1981), “Bregman’s balancing method,” Transport. Res. 15B, pp. 239–248.
Lent, A. (1977), “A convergent algorithm for maximum entropy image restoration with a medical x-ray application,” in Image Analysis and Evaluation, R. Shaw, ed., Society of Photographic Scientists and Engineers ( SPSE ), Washington, D.C., pp. 238–243.
Lent, A., and Y. Censor (1980), “Extensions of Hildredth’s row-action method for quadratic programming” SIAM J. on Control and Optimization 18, pp. 444–454.
Levine, R. D., and M. Tribus, eds. (1978), The Maximum Entropy Formalism, MIT Press, Cambridge, Mass.
Minerbo, G. N. (1979), “MENT: a maximum entropy algorithm for reconstructing a source from projection data,” Comput. Graphics and Image Process. 10, pp. 48–68.
Minerbo, G. N., and J. G. Sanderson (1977), “Reconstruction of a source from a few (2 or 3) projections,” Tech. Rept. LA-6747-MS, Los Alamos Scientific Laboratories, New Mexico.
Wong, D. S. (1981), “Maximum likelihood, entropy maximization, and the geometric programming approaches to the calibration of trip distribution models,” Transport. Res. 15B, pp. 329–343.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1987 D. Reidel Publishing Company
About this paper
Cite this paper
Censor, Y., Elfving, T., Herman, G.T. (1987). Special-Purpose Algorithms for Linearly Constrained Entropy Maximization. In: Smith, C.R., Erickson, G.J. (eds) Maximum-Entropy and Bayesian Spectral Analysis and Estimation Problems. Fundamental Theories of Physics, vol 21. Springer, Dordrecht. https://doi.org/10.1007/978-94-009-3961-5_14
Download citation
DOI: https://doi.org/10.1007/978-94-009-3961-5_14
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-010-8257-0
Online ISBN: 978-94-009-3961-5
eBook Packages: Springer Book Archive