Abstract
Hidden Conditional Random Fields (HCRFs) are discriminative latent variable models which have been shown to successfully learn the hidden structure of a given classification problem. An infinite HCRF is an HCRF with a countably infinite number of hidden states, which rids us not only of the necessity to specify a priori a fixed number of hidden states available but also of the problem of overfitting. Markov chain Monte Carlo (MCMC) sampling algorithms are often employed for inference in such models. However, convergence of such algorithms is rather difficult to verify, and as the complexity of the task at hand increases, the computational cost of such algorithms often becomes prohibitive. These limitations can be overcome by variational techniques. In this paper, we present a generalized framework for infinite HCRF models, and a novel variational inference approach on a model based on coupled Dirichlet Process Mixtures, the HCRF–DPM. We show that the variational HCRF–DPM is able to converge to a correct number of represented hidden states, and performs as well as the best parametric HCRFs —chosen via cross–validation— for the difficult tasks of recognizing instances of agreement, disagreement, and pain in audiovisual sequences.
Chapter PDF
Similar content being viewed by others
Keywords
References
Rasmussen, C.: The Infinite Gaussian Mixture Model. In: Proc. Advances in Neural Information Processing Systems, pp. 554–560 (2000)
Beal, M., Ghahramani, Z., Rasmussen, C.: The Infinite Hidden Markov Model. In: Proc. Advances in Neural Information Processing Systems, pp. 577–584 (2002)
Fox, E., Sudderth, E., Jordan, M., Willsky, A.S.: An HDP–HMM for systems with state persistence. In: Proc. Int’l Conf. on Machine Learning (2008)
Orbanz, P., Buhmann, J.: Nonparametric Bayes Image Segmentation. Int’l Journal in Computer Vision 77, 25–45 (2008)
Van Gael, J., Teh, Y., Ghahramani, Z.: The Infinite Factorial Hidden Markov Model. Advances in Neural Information Processing Systems 21, 1697–1704 (2009)
Chatzis, S., Tsechpenakis, G.: The infinite hidden Markov random field model. IEEE Trans. Neural Networks 21(6), 1004–1014 (2010)
Bousmalis, K., Zafeiriou, S., Morency, L.P., Pantic, M.: Infinite hidden conditional random fields for human behavior analysis. IEEE Trans. Neural Networks and Learning Systems 24(1), 170–177 (2013)
Ghahramani, Z., Beal, M.: Propagation Algorithms for Variational Bayesian Learning. Advances in Neural Information Processing Systems 13, 507–513 (2001)
Sethuraman, J.: A Constructive Definition of Dirichlet Priors. Statistica Sinica 4, 639–650 (1994)
Quattoni, A., Wang, S., Morency, L., Collins, M., Darrell, T.: Hidden Conditional Random Fields. IEEE Trans. Pattern Analysis and Machine Intelligence, 1848–1852 (2007)
Blei, D., Jordan, M.: Variational Inference for Dirichlet Process Mixtures. Bayesian Analysis 1(1), 121–144 (2006)
Bertsekas, D.: On the Goldstein-Levitin-Polyak Gradient Projection Method. IEEE Trans. on Automatic Control 21, 174–184 (1976)
Bertsekas, D.: Nonlinear Programming. Athena Scientific (1999)
Vinciarelli, A., Pantic, M., Bourlard, H.: Social Signal Processing: Survey of an Emerging Domain. Image and Vision Computing 27(12), 1743–1759 (2009)
Ekman, P., Friesen, W.V., Hager, J.C.: Facial Action Coding System. Research Nexus, Salt Lake City (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bousmalis, K., Zafeiriou, S., Morency, L., Pantic, M., Ghahramani, Z. (2013). Variational Hidden Conditional Random Fields with Coupled Dirichlet Process Mixtures. In: Blockeel, H., Kersting, K., Nijssen, S., Železný, F. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2013. Lecture Notes in Computer Science(), vol 8189. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40991-2_34
Download citation
DOI: https://doi.org/10.1007/978-3-642-40991-2_34
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-40990-5
Online ISBN: 978-3-642-40991-2
eBook Packages: Computer ScienceComputer Science (R0)