Abstract
The standard approach for learning Markov Models with Hidden State uses the Expectation-Maximization framework. While this approach had a significant impact on several practical applications (e.g. speech recognition, biological sequence alignment) it has two major limitations: it requires a known model topology, and learning is only locally optimal. We propose a new PAC framework for learning both the topology and the parameters in partially observable Markov models. Our algorithm learns a Probabilistic Deterministic Finite Automata (PDFA) which approximates a Hidden Markov Model (HMM) up to some desired degree of accuracy. We discuss theoretical conditions under which the algorithm produces an optimal solution (in the PAC-sense) and demonstrate promising performance on simple dynamical systems.
Chapter PDF
Similar content being viewed by others
References
Carrasco, R., Oncina, J.: Learning stochastic regular grammars by means of a state merging method. In: Carrasco, R.C., Oncina, J. (eds.) ICGI 1994. LNCS, vol. 862. Springer, Heidelberg (1994)
Clark, A., Thollard, F.: PAC-learnability of Probabilistic Deterministic Finite State Automata. Journal of Machine Learning Research 5 (2004)
Dupont, P., Denis, F., Esposito, Y.: Links between Probabilistic Automata and Hidden Markov Models. Pattern Recognition 38(9) (2005)
Durbin, R., Eddy, S.R., Krogh, A., Mitchison, G.J.: Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids. Cambridge University Press, Cambridge (1998)
Holmes, M., Isbell, C.: Looping Suffix Tree-Based Inference of Partially Observable Hidden State. In: Proceedings of ICML (2006)
Jaeger, H., Zhao, M., Kolling, A.: Efficient estimation of OOMs. In: Proceedings of NIPS (2005)
Kearns, M., Mansour, Y., Ron, D., Rubinfeld, R., Schapire, R.E., Sellie, L.: On the learnability of discrete distributions. In: ACM Symposium on the Theory of Computing (1995)
Lipton, R.J., Naughton, J.F.: Query size estimation by adaptive sampling. J. Computer and System Sciences 51, 18–25 (1995)
Ostendorf, M., Singer, H.: HMM topology design using maximum likelihood successive state splitting. Computer Speech and Language 11 (1997)
Rabiner, L.R.: A tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceedings of the IEEE 77(2) (1989)
Ron, D., Singer, Y., Tishby, N.: On the learnability and usage of acyclic robabilistic finite automata. In: Proceedings of COLT (1995)
Rosencrantz, M., Gordon, G., Thrun, S.: Learning Low Dimensional Predictive Representations. In: Proceedings of ICML (2004)
Singh, S., Littman, M.L., Jong, N.K., Pardoe, D., Stone, P.: Learning Predictive State Representations. In: Proceedings of ICML (2003)
Stolcke, A., Omohundro, S.M.: Hidden Markov Model Induction by Bayesian Model Merging. In: Proceedings of NIPS (1993)
Thollard, F., Dupont, P., de la Higuera, C.: Probabilistic DFA Inference using Kullback-Leibler Divergence and Minimality. In: Proceedings of ICML (2000)
Valiant, L.: A theory of the learnable. Communications of the ACM 27(11) (1984)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gavaldà, R., Keller, P.W., Pineau, J., Precup, D. (2006). PAC-Learning of Markov Models with Hidden State. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds) Machine Learning: ECML 2006. ECML 2006. Lecture Notes in Computer Science(), vol 4212. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11871842_18
Download citation
DOI: https://doi.org/10.1007/11871842_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-45375-8
Online ISBN: 978-3-540-46056-5
eBook Packages: Computer ScienceComputer Science (R0)