Skip to main content
SpringerLink
Log in

Extensions and Related Results

  • Chapter
Entropy Optimization and Mathematical Programming

Part of the book series: International Series in Operations Research & Management Science ((ISOR,volume 8))

  • 265 Accesses

Abstract

This chapter discusses extensions of the methods developed in previous chapters and discusses some closely related subjects, including

  1. (i)

    Entropy optimization problems with a finite number of constraints but a countably infinite number of variables,

  2. (ii)

    A relationship between entropy optimization and Bayesian statistical estimation,

  3. (iii)

    Entropic regularization method for solving min-max problems, and

  4. (iv)

    Entropic regularization method for solving semi-infinite min-max problems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Apostol, A.M., Mathematical Analysis, Addison-Wesley Publishing, Redwood City, CA, 1977.

    Google Scholar 

  2. Ben-Tal, A., and Charnes, A., “A Dual Optimization Framework for Some Problems of Information Theory and Statistics,” Problems of Control and Information Theory, Vol. 8, 1979, pp. 387–401.

    Google Scholar 

  3. Ben-Tal, A., and Teboulle, M., “A Smooth Technique for Nondifferentiable Optimization Problems,” Optimization — Fifth French-German Conference, Castel Novel, 1988, Lecture Notes in Mathematics 1405, Springer Verlag, 1989, pp. 1-11.

    Google Scholar 

  4. Ben-Tal, A., Teboulle, M., and Charnes, A., “The Role of Duality in Optimization Problems involving Entropy Functionals with Applications to Information Theory,” Journal of Optimization Theory and Applications, Vol. 58, 1988, pp. 209–223.

    Article  Google Scholar 

  5. Bertsekas, D., Constrained Optimization and Lagrange Multiplier Methods, Academic Press, New York, 1989.

    Google Scholar 

  6. Billingsley, P., Probability and Measure, John Wiley, New York, 1979.

    Google Scholar 

  7. van Campenhout, J. and Cover, T.M., “Maximum Entropy and Conditional Probability,” IEEE Transactions on Information Theory, Vol. 27, 1981, pp. 483–489.

    Article  Google Scholar 

  8. Charalambous, C., and Conn, A.R., “An Efficient Method to Solve the Minimax Problem Directly,” SIAM Journal on Numerical Analysis, Vol. 15, 1978, pp. 162–187.

    Article  Google Scholar 

  9. Charnes, A., Cooper, W.W., and Seiford, L., “Extremal Principles and Optimization Qualities for Khinchin-Kullback-Leibler Estimation,” Mathematische Operationsforschung und Statistik, Series Optimization, Vol. 9, 1978, pp. 21–29.

    Article  Google Scholar 

  10. DeGroot, M.H., Optimal Statistical Decisions, McGraw-Hill, New York, 1970.

    Google Scholar 

  11. Di Pillo, G., Grippo, L., and Lucidi, S., “A Smooth Method for the Finite Minimax Problem,” Mathematical Programming, Vol. 60, 1993, pp. 187–214.

    Article  Google Scholar 

  12. Fang, S.-C., and Rajasekera, J.R., “Quadratically Constrained Minimum Cross-Entropy Analysis,” Mathematical Programming, Vol. 44, 1989, pp. 85–96.

    Article  Google Scholar 

  13. Fang, S.-C., and Puthenpura, S., Linear Optimization and Extensions: Theory and Algorithms, Prentice Hall, Englewood Cliffs, New Jersey, 1993.

    Google Scholar 

  14. Fang, S.-C., and Wu, S.-Y., “Solving Min-Max Problems and Linear Semi-Infinite Programs,” Computers and Mathematics with Applications, Vol. 32, 1996, pp. 87–93.

    Article  Google Scholar 

  15. Fletcher, R., Practical Methods of Optimization, Vol. 2, John Wiley, New York, 1981.

    Google Scholar 

  16. Gigola, C., and Gomez, S., “A Regularization Method for Solving the Finite Convex Min-Max Problem,” SIAM Journal on Numerical Analysis, Vol. 27, 1990, pp. 1621–1634.

    Article  Google Scholar 

  17. Hettich, R., and Kortanek, K.O., “Semi-Infinite Programming: Theory, Method and Applications,” SIAM Review, Vol. 35, 1993, pp. 380–429.

    Article  Google Scholar 

  18. Huard, P., “Resolution of Mathematical Programming with Nonlinear Constraints by the Method of Centers,” in Nonlinear Programming, edited by J. Abadie, North-Holland, Amsterdam, 1967, pp. 207–219.

    Google Scholar 

  19. Hiriart-Urruty, J.-B., and Lemarechal, C., Convex Analysis and Minimization Algorithm, Springer-Verlag, Berlin, 1993.

    Google Scholar 

  20. Li, X.S., and Fang, S.-C., “On the Entropic Regularization Method for Solving Min-Max Problems with Applications,” to appear in Mathematical Methods of Operations Research, Vol. 46, 1997.

    Google Scholar 

  21. Jaynes, E.T., “Information Theory and Statistical Mechanics,” Physics Review, Vol. 106, 1957, pp. 620–630.

    Article  Google Scholar 

  22. Jaynes, E.T., “Information Theory and Statistical Mechanics II,” Physics Review, Vol. 108, 1957, pp. 171–190.

    Article  Google Scholar 

  23. Jaynes, E.T., “Prior Probabilities,” IEEE Transactions. Systems Science and Cybernetics, Vol. 4, 1968, pp. 227–241.

    Article  Google Scholar 

  24. Jaynes, E.T., “The Relation of Bayesian and Maximum Entropy Methods,” Maximum-Entropy and Bayesian Methods in Science and Engineering, Volume 1: Foundations, edited by G. J. Erickson and C. R. Smith, Kluwer Academic Publishers, Dordrecht, The Netherlands, 1988, pp. 25–29.

    Chapter  Google Scholar 

  25. Kazarinoff, N.D., Analytic Inequalities, Holt, Rinehart and Winston, New York, 1961.

    Google Scholar 

  26. Kullback, S., Information and Statistics, John Wiley, New York, 1959.

    Google Scholar 

  27. Li, X.-S., Entropy and Optimization, Ph.D. Thesis, University of Liverpool, United Kingdom, 1987.

    Google Scholar 

  28. Lin, C.J., Fang, S.-C., and Wu, S.Y., Parametric Linear Semi-Infinite Programming, Applied Mathematics Letters, Vol. 9, 1996, pp. 89–96.

    Article  Google Scholar 

  29. Lin, C.J., Fang, S.-C., and Wu, S.Y., An Unconstrained Convex Programming Approach to Linear Semi-Infinite Programming, OR Technical Report No. 296, North Carolina State University, Raleigh, North Carolina, 1994, submitted to SIAM Journal on Optimization.

    Google Scholar 

  30. Mishra, S., and Fang, S.-C., “A Maximum Entropy Optimization Approach to Tandem Queues with Generalized Blocking,” Performance Evaluation, Vol. 702, 1997, pp. 1–25.

    Google Scholar 

  31. Oko, S.O., “Surrogate Methods for Linear Inequalities,” Journal of Optimization Theory and Applications, Vol. 72, 1992, pp. 247–268.

    Article  Google Scholar 

  32. Polak, E., Mayne, D.Q., and Higgins, J.E., “Superlinearly Convergent Algorithm for Min-Max Problems,” Journal of Optimization Theory and Applications, Vol. 69, 1991, pp. 407–439.

    Article  Google Scholar 

  33. Polyak, R.A., “Smooth Optimization Methods for Minimax Problems,” SIAM Journal of Control and Optimization, Vol. 26, 1988, pp. 1274–1286.

    Article  Google Scholar 

  34. Rajasekera, J.R., and Fang, S.-C., “Deriving an Unconstrained Convex Program for Linear Programming,” Journal of Optimization Theory and Applications, Vol. 75, 1992, pp. 603–612.

    Article  Google Scholar 

  35. Royden, H.L., Real Analysis, 2nd Edition, The Macmillan Company, New York, 1972.

    Google Scholar 

  36. Rudin, W., Principles of Mathematical Analysis, McGraw Hill, New York, 1976.

    Google Scholar 

  37. Tsao, H.-S.J., Fang, S.-C., and Lee, D.N., “On the Optimal Entropy Analysis,” European Journal of Operational Research, Vol. 59, 1992, pp. 324–329.

    Article  Google Scholar 

  38. Tsao, H.-S.J., Fang, S.-C., and Lee, D.N., “A Bayesian Interpretation of the Linearly-Constrained Cross-Entropy Minimization Problem,” Engineering Optimization, Vol. 22, 1993, pp. 65–75.

    Article  Google Scholar 

  39. Vardi, A., “New Minmax Algorithm,” Journal of Optimization Theory and Applications, Vol. 75, 1992, pp. 613–633.

    Article  Google Scholar 

  40. Williams, P.M., “Bayesian Conditionalisation and the Principle of Minimum Information,” The British Journal for the Philosophy of Science, Vol. 31, 1980, pp. 131–144.

    Article  Google Scholar 

  41. Wu, J.-S., and Chan, W.C., “Maximum Entropy Analysis of Multiple-Server Queueing Systems,” Journal of Operational Research Society, Vol. 40, 1989, pp. 815–826.

    Google Scholar 

  42. Yang, K., and Murty, K.G., “New Iterative Methods for Linear Inequalities,” Journal of Optimization Theory and Applications, Vol. 72, 1992, pp. 163–185.

    Article  Google Scholar 

  43. Zang, I., “A Smoothing Out Technique for Min-Max Optimization,” Mathematical Programming, Vol. 19, 1980, pp. 61–77.

    Article  Google Scholar 

  44. Zellner, A., “Optimal Information Processing and Bayes’ Theorem,” The American Statistician, Vol. 42, 1988, pp. 278–284.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. North Carolina State University, Raleigh, North Carolina, USA

    S.-C. Fang

  2. International University of Japan, Urasa, Niigata, Japan

    J. R. Rajasekera

  3. University of California, Berkeley, California, USA

    H.-S. J. Tsao

Authors
  1. S.-C. Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. R. Rajasekera
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. H.-S. J. Tsao
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 1997 Springer Science+Business Media New York

About this chapter

Cite this chapter

Fang, SC., Rajasekera, J.R., Tsao, HS.J. (1997). Extensions and Related Results. In: Entropy Optimization and Mathematical Programming. International Series in Operations Research & Management Science, vol 8. Springer, Boston, MA. https://doi.org/10.1007/978-1-4615-6131-6_7

Download citation

  • DOI: https://doi.org/10.1007/978-1-4615-6131-6_7

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-1-4613-7810-5

  • Online ISBN: 978-1-4615-6131-6

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation