Skip to main content
SpringerLink
Log in

Mathematical Foundations of Hidden Markov Models

  • Conference paper
Recent Advances in Speech Understanding and Dialog Systems

Part of the book series: NATO ASI Series ((NATO ASI F,volume 46))

Abstract

Stochastic methods of signal modeling have become increasingly popular. There are two strong reasons why this has occurred. First the models are very rich in mathematical structure and hence can form the theoretical basis for use in a wide range of applications. Second the models, when applied properly, work very well in practice for several important applications. In this paper we attempt to carefully and methodically review the theoretical aspects of one type of stochastic modelling, namely hidden Markov models (HMM’s), and show how they have been applied to a couple of problems in machine recognition of speech.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Bibliography On Hmm’s

Markov Processes

  1. A. W. Drake, “Discrete - State Markov Processes,” Chapter 5 in Fundamentals of Applied Probability Theory, McGraw Hill Book Co, 1967.

    Google Scholar 

Hidden Markov Model Theory

  1. L. E. Baum, T. Petrie, G. Soules, and N. Weiss, “A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains,” Ann. Math. Stat., Vol. 41, No. 1, pp. 164–171, 1970.

    Article  MathSciNet  MATH  Google Scholar 

  2. L. E. Baum and J. A. Egon, “An Inequality with Applications to Statistical Estimation for Probabilistic Functions of a Markov Process and to a Model for Ecology,” Bull. AMS, Vol. 73, pp. 360–363, 1967.

    Article  MATH  Google Scholar 

  3. L. E. Baum and T. Petrie, “Statistical Inference for Probabilistic Functions of Finite State Markov Chains,” Ann. Math. Stat., Vol. 37, pp. 1554–1563, 1966.

    Article  MathSciNet  MATH  Google Scholar 

  4. L. E. Baum and G. R. Sell, “Growth Functions for Transformations on Manifolds,” Pac. J. Math., Vol. 27, No. 2, pp. 211–227, 1968.

    MathSciNet  MATH  Google Scholar 

  5. L. E. Baum, “An Inequality and Associated Maximization Technique in Statistical Estimation for Probabilistic Functions of Markov Processes,” Inequalities, Vol. 3, pp. 1–8, 1972.

    Google Scholar 

  6. A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum Likelihood from Incomplete Data Via the EM Algorithm,” J. Royal Stat. Soc., Vol. 39, No. 1, pp. 1–38, 1977.

    MathSciNet  MATH  Google Scholar 

  7. L. R. Rabiner and B. H. Juang, “An Introduction to Hidden Markov Models,” IEEE ASSP Magazine, Vol. 3, No. 1, pp. 4–16, Jan. 1986.

    Article  Google Scholar 

  8. S. E. Levinson, L. R. Rabiner, and M. M. Sondhi, “An Introduction to the Application of the Theory of Probabilistic Functions of a Markov Process to Automatic Speech Recognition,” Bell System Tech. J., Vol. 62, No. 4, pp. 1035–1074, April 1983.

    MathSciNet  MATH  Google Scholar 

  9. L. A. Liporice, “Maximum Likelihood Estimation for Multivariate Observations of Markov Sources,” IEEE Trans, on Information Theory, Vol./ IT-28, No. 5, pp. 729–734, 1982.

    Article  Google Scholar 

  10. B. H. Juang, “On the Hidden Markov Model and Dynamic Time Warping for Speech Recognition - A Unified View,” AT&T Tech. J., Vol. 63, No. 7, pp. 1213–1243, September 1984.

    MathSciNet  MATH  Google Scholar 

  11. B. H. Juang, S. E. Levinson, and M. M. Sondhi, “Maximum Likelihood Estimation for Multivariate Mixture Observations of Markov Chains,” IEEE Trans, on Information Theory, Vol. IT-32, No. 2, pp. 307–309, March 1986.

    Article  Google Scholar 

  12. B. H. Juang, “Maximum Likelihood Estimation for Mixture Multivariate Stochastic Observations of Markov Chains,” AT&T Tech. J., Vol. 64, No. 6, pp. 1235–1249, July-Aug. 1985.

    MathSciNet  MATH  Google Scholar 

  13. L. R. Rabiner, B. H. Juang, S. E. Levinson, and M. M. Sondhi, “Some Properties of Continuous Hidden Markov Model Representations,” AT&T Tech. J., Vol. 64, No. 6, pp. 1251–1270, July-Aug. 1985.

    MathSciNet  Google Scholar 

  14. A. B. Poritz, “Linear Predictive Hidden Markov Models and the Speech Signal,” Proc. ICASSP ‘82, Paris, France, pp. 1291–1294, May 1982.

    Google Scholar 

  15. A. H. Juang and L. R. Rabiner, “Mixture Autoregressive Hidden Markov Models for Speech Signals,” IEEE Trans, on Acoustics, Speech, and Signal Proc., Vol. ASSP-33, No. 6, pp. 1404–1413, Dec. 1985.

    Article  MathSciNet  Google Scholar 

  16. S. E. Levinson, “Continuously Variable Duration Hidden Markov Models for Automatic Speech Recognition,” Computer, Speech and Language, Vol. 1, No. 1, pp. 29–45, March 1986.

    Article  Google Scholar 

  17. M. J. Russell and R. K. Moore, “Explicit Modeling of State Occupancy in Hidden Markov Models for Automatic Speech Recognition,” Proc. ICASSP-85, Tampa, Florida, pp. 5–8, March 1985.

    Google Scholar 

  18. L. R. Bahl, P. F. Brown, P. V. de Souza, and R. L. Mercer, “Maximum Mutual Information Estimation of Hidden Markov Model Parameters for Speech Recognition,” Proc. ICASSP-86, Tokyo, Japan, pp. 49–52, April 1986.

    Google Scholar 

  19. Y. Ephraim, A. Dembo, and L. R. Rabiner, “A Minimum Discrimination Information Approach for Hidden Markov Modeling,” Proc. ICASSP-87, Dallas, Texas, April 1987.

    Google Scholar 

  20. F. Jelinek and R. L. Mercer, “Interpolated Estimation of Markov Source Parameters from Sparse Data,” in Pattern Recognition in Practice, E. S. Gelesma and L. N. Kanal, Eds., North Holland, pp. 381–402, 1980.

    Google Scholar 

  21. B. H. Juang and L. R. Rabiner, “A Probabilistic Distance Measure for Hidden Markov Models,” AT&T Tech. J., Vol. 64, No. 2, pp. 391–408, Feb. 1985.

    MathSciNet  Google Scholar 

General Speech Recognition with VQ and HMM’s

  1. S. E. Levinson, “Structural Methods in Automatic Speech Recognition,” Proc. IEEE, Vol. 73, No. 11, pp. 1625–1650, Nov. 1985.

    Article  Google Scholar 

  2. J. Makhoul, S. Roucos, and H. Gish, “Vector Quantization in Speech Coding,” Proc. IEEE, Vol. 73, No. 11, pp. 1551–1588, Nov. 1985.

    Article  Google Scholar 

  3. J. S. Bridle, “Stochastic Models and Template Matching: Some Important Relationships Between Two Apparently Different Techniques for Automatic Speech Recognition,” Proc. Inst, of Acoustics, Autum Conf., pp. 1–8, Nov. 1984.

    Google Scholar 

Isolated Word Recognition Using HMM’s

  1. L. R. Rabiner, S. E. Levinsion, and M. M. Sondhi, “On the Application of Vector Quantization and Hidden Markov Models to Speaker-Independent Isolated Word Recognition,” Bell System Tech. J., Vol. 62, No. 4, pp. 1075–1105, April 1983.

    Google Scholar 

  2. L. R. Rabiner, S. E. Levinson, and M. M. Sondhi, “On the Use of Hidden Markov Models for Speaker-Independent Recognition of Isolated Words from a Medium-Size Vocabulary,” AT&T Tech. Jourol. 63, No. 4, pp. 627–642, April 1984.

    Google Scholar 

  3. R. Billi, “Vector Quantization and Markov Source Models Applied to Speech Recognition,” Proc. ICASSP-82, Paris, France, pp. 574–577, May 1982.

    Google Scholar 

  4. L. R. Rabiner, B. H. Juang, S. E. Levinson, and M. M. Sondhi, “Recognition of Isolated Digits Using Hidden Markov Models with Continuous Mixture Densities,” AT&T Tech. J., Vol. 64, No. 6, pp. 1211–1222, July-Aug. 1986.

    MathSciNet  Google Scholar 

  5. A. B. Poritz and A. G. Richter, “Isolated Word Recognition,” Proc. ICASSP-86, Tokyo, Japan, pp. 705–708, April 1986.

    Google Scholar 

Connected Word Recognition Using HMM’s

  1. L. R. Rabiner, J. G. Wilpon, and B. H. Juang, “A Segmental k-Means Training Procedure for Connected Word Recognition,” AT&T Tech. J., Vol. 65, No. 3, pp. 21–31, May-June 1986.

    Google Scholar 

  2. L. R. Rabiner and S. E. Levinson, “A Speaker-Independent, Syntax-Directed, Connected Word Recognition System Based on Hidden Markov Models and Level Building,” IEEE Trans, on Acoustics, Speech, and Signal Processing, Vol. ASSP-33, No. 3, pp. 561–573, June 1985.

    Article  Google Scholar 

  3. L. R. Rabiner, J. G. Wilpon, and B. H. Juang, “A Model-Based Connected Digit Recognition System Using Either Hidden Markov Models or Templates,” Computer, Speech, and Language, 1986.

    Google Scholar 

  4. H. Bourlard, Y. Kamp, H. Ney, and C. J. Wellekens, “Speaker-Dependent Connected Speech Recognition via Dynamic Programming and Statistical Methods,” in Speech and Speaker Recognition, M. R. Schroeder, Ed., Karger, pp. 115–148, 1985.

    Google Scholar 

  5. C. J. Wellekens, “Global Connected Digit Recognition Using Baum-Welch Algorithm,” Proc. ICASSP-86, Tokyo, Japan, pp. 1081–1084, April 1986.

    Google Scholar 

Continuous Speech Recognition Using HMM’s

  1. L. R. Bahl, F. Jelinek, and R. L. Mercer, “A Maximum Likelihood Approach to Continuous Speech Recongition,” IEEE Trans, on Pattern Anal, and Machine Intell., Vol. PAMI-5, pp. 179–190, 1983.

    Article  Google Scholar 

  2. F. Jelink, “Continuous Speech Recognition by Statistical Methods,” Proc. IEEE, Vol. 64, pp. 532–536, April 1976.

    Article  Google Scholar 

  3. J. K. Baker, “The Dragon System - An Overview,” IEEE Trans, on Acoustics, Speech, and Signal Processing, Vol. ASSP-23, No. 1, pp. 24–29, Feb. 1975.

    Article  Google Scholar 

  4. L. R. Bahl and F. Jelinek, “Decoding for Channels with Insertions, Deletions, and Substitutions with Applications to Speech Recognition,” IEEE Trans. Information Theory, Vol. IT-21, pp. 404–411, 1975.

    Article  Google Scholar 

  5. F. Jelinek, “A Fast Sequential Decoding Algorithm Using a Stack,” IBM J. Res. and Devel, Vol. 13, pp. 675–685, 1969.

    Article  MathSciNet  MATH  Google Scholar 

  6. F. Jelinek, L. R. Bahl, and R. L. Mercer, “Continuous Speech Recognition: Statistical Methods,” in Handbook of Statistics, II, P. R. Krishnaiad, Ed., North Holland, 1982.

    Google Scholar 

  7. F. Jelinek, L. R. Bahl, and R. L. Mercer, “Design of a Linguistic Statistical Decoder for the Recognition of Continuous Speech,” IEEE Trans, on Information Theory, Vol. IT-21, pp. 250–256, 1975.

    Article  Google Scholar 

  8. S. E. Levinson, “Continuous Speech Recognition by Means of Acoustic-Phonetic Classification Obtained from a Hidden Markov Model,” Proc. ICASSP-87, Dallas, Texas, April 1987.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. AT&T Bell Laboratories, 07974, Murray Hill, New Jersey, USA

    Lawrence R. Rabiner

Authors
  1. Lawrence R. Rabiner
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Universität Erlangen-Nürnberg, Martensstr. 3, D-8520, Erlangen, Germany

    H. Niemann  & G. Sagerer  & 

  2. ZT ZTI SYS 5, Siemens AG, Otto-Hahn-Ring 6, D-8000, München 83, Germany

    M. Lang

Rights and permissions

Reprints and permissions

Copyright information

© 1988 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Rabiner, L.R. (1988). Mathematical Foundations of Hidden Markov Models. In: Niemann, H., Lang, M., Sagerer, G. (eds) Recent Advances in Speech Understanding and Dialog Systems. NATO ASI Series, vol 46. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-83476-9_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-83476-9_19

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-83478-3

  • Online ISBN: 978-3-642-83476-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation