Bayesian Estimation and Classification

  • Saeed V. Vaseghi


Bayesian estimation is a framework for formulation of statistical inference problems, and includes the classical estimators such as the maximum a posterior, maximum likelihood, minimum mean squared error, and minimum mean absolute value of error as its special cases. The hidden Markov model, widely used in statistical signal processing, is also an example of a Bayesian model. Bayesian inference is based on the minimisation of a so called Bayes risk function which includes; a posterior model of the unknown parameters given the observation, and a cost of error function. This chapter begins with an introduction to the basic concepts of estimation theory, and considers the statistical measures that are used to quantify the performance of an estimator. We study the Bayesian estimation methods and consider the effects of using a prior model on the mean and the variance of an estimate. Estimation of discrete-valued parameters, and parameters from a finite-state process, are studied within the frame work of Bayesian classification. The chapter concludes with a study of the methods for the modelling of a random signal space.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Andergerg M.R. (1973), Cluster Analysis for Applications, Academic Press, New York.Google Scholar
  2. Abramson, N. (1963), Information Theory and Coding, McGraw Hill, New York.Google Scholar
  3. Baum L.E., Petrie T., Soules G. Weiss N. (1970), A Maximisation Technique occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains, Ann. Math. Stat. Vol. 41, pages 164–171.MathSciNetzbMATHCrossRefGoogle Scholar
  4. Bayes T. (1763) An Essay Towards Solving a Problem in the Doctrine of Changes, Philosophical Transactions of the Royal Society of London, Vol. 53, pages 370–418. and reprinted in 1958 in Biometrika Vol.45, pages 293–315.Google Scholar
  5. Chou P. Lookabaugh T. and Gray R. (1989), Entropy-Constrained Vector Quantisation, IEEE Trans. Acoustics, Speech and Signal Processing, Vol. ASSP-37, pages 31–42.MathSciNetGoogle Scholar
  6. Bezdek J.C. (1981), Pattern Recognition with Fuzzy Objective Function Algorithms, Plenum Press, New York.zbMATHCrossRefGoogle Scholar
  7. Cramer H. (1974), Mathematical Methods of Statistics. Princeton University Press, New Jersey.Google Scholar
  8. Deutsch R. (1965), Estimation Theory, Prentice-Hall, Englewood Cliffs, N. J.zbMATHGoogle Scholar
  9. Dempster A.P., Laird N.M., Rubin D.B. (1977), Maximum Likelihood from Incomplete Data via the EM Algorithm, J. Royal Stat. Soc. ser. B Vol. 39, Pages 1–38.MathSciNetzbMATHGoogle Scholar
  10. Duda R.O. Hart R.E. (1973), Pattern Classification, Wiley, New York.zbMATHGoogle Scholar
  11. Feder M., Weinstein E. (1988), Parameter Estimation of Superimposed Signals using the EM algorithm, IEEE Trans. Acoustics, Speech and Signal Processing, Vol. ASSP-36(4), pages 477.489.CrossRefGoogle Scholar
  12. Fisher R. A. (1922), On the Mathematical Foundations of the Theoretical Statistics, Phil Trans. Royal. Soc. London, Vol. 222, Pages 309–368.Google Scholar
  13. Gersho A. (1982), On the Structure of Vector Quantisers, IEEE Trans. Information Theory, Vol. IT-28, pages 157–166.Google Scholar
  14. Gray R.M. (1984), Vector Quantisation, IEEE ASSP Magazine pages 4–29.Google Scholar
  15. Gray R.M., Karnin E.D, Multiple local Optima in Vector Quantisers, IEEE Trans. Information Theory, Vol. IT-28, pages 256–261.Google Scholar
  16. Jeffrey H. (1961), Scientific Inference, 3rd ed. Cambridge University Press.Google Scholar
  17. Larson H.J and Bruno O.S. (1979), Probabilistic Models in Engineering Sciences, Vol. I and II. Wiley, New York.Google Scholar
  18. Linde Y., Buzo A., Gray R.M. (1980), An Algorithm for Vector Quantiser Design, IEEE Trans. Comm. Vol. COM-28, pages 84–95.CrossRefGoogle Scholar
  19. Makhoul J., Roucos S., Gish H. (1985), Vector Quantisation in Speech Coding, Proc. IEEE, Vol. 73, pages 1551–1588.CrossRefGoogle Scholar
  20. Mohanty N. (1986), Random Signals, Estimation and Identification, Van Nostrand, New York.CrossRefGoogle Scholar
  21. Rao C.R. (1945), Information and Accuracy Attainable in the Estimation of Statistical Parameters, Bull Calcutta Math. Soc, Vol. 37, Pages 81–91.MathSciNetzbMATHGoogle Scholar
  22. Render R.A., Walker H.F. (1984), Mixture Densities, Maximum Likelihood and the EM algorithm, SIAM review, Vol. 26, Pages 195–239.MathSciNetCrossRefGoogle Scholar
  23. Scharf L.L. (1991), Statistical Signal Processing: Detection, Estimation, and Time Series Analysis, Addison Wesley, Reading, Mass.zbMATHGoogle Scholar

Copyright information

© John Wiley & Sons Ltd. and B.G. Teubner 1996

Authors and Affiliations

  • Saeed V. Vaseghi
    • 1
  1. 1.Queen’s UniversityBelfastUK

Personalised recommendations