Abstract
Gaussian mean field is an important paradigm of cluster-based variational inference, and its cluster selection is critical to the tradeoff between the variational accuracy and the computational complexity of cluster-based variational inference. In this paper, we explore a coupling based cluster selection method for Gaussian mean fields. First, we propose the model coupling and the quasi-coupling concepts on Gaussian Markov random field, and prove the coupling-accuracy theorem for Gaussian mean fields, which regards the quasi-coupling as a cluster selection criterion. Then we design a normalized cluster selection algorithm based on the criterion for Gaussian mean fields. Finally, we design numerical experiments to demonstrate the validity and efficiency of the cluster selection method and algorithm.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Speed, T., Kiiveri, H.: Gaussian Markov Distributions over Finite Graphs. Annals of Statistics 14, 138–150 (1986)
Rue, H., Held, L.: Gaussian Markov Random Fields: Theory and Applications. Chapman & Hall, London (2005)
Saul, L., Jordan, M.: Exploiting Tractable Substructures in Intractable Networks. In: Advances in Neural Information Processing Systems, vol. 8, pp. 486–492. MIT Press, Cambridge (1996)
Wiegerinck, W.: Variational Approximations between Mean Field Theory and the Junction Tree Algorithm. In: 16th Conference on Uncertainty in Artificial Intelligence, pp. 626–633. Morgan Kaufmann, San Francisco (2000)
Welling, M.: On the Choice of Regions for Generalized Belief Propagation. In: 20th Conference on Uncertainty in Artificial Intelligence, pp. 585–592. AUAI Press, Arlington (2004)
Welling, M., Minka, T.P., Teh, Y.W.: Structured Region Graphs: Morphing EP into GBP. In: 21th Conference on Uncertainty in Artificial Intelligence, pp. 609–619. AUAI Press, Arlington (2005)
Xing, E., Jordan, M., Russell, S.: Graph Partition Strategies for Generalized Mean Field Inference. In: 20th Conference on Uncertainty in Artificial Intelligence, pp. 602–610. AUAI Press, Arlington (2004)
Wainwright, M., Jordan, M.: Graphical Models, Exponential Families, and Variational Inference. Technical Report 649, Department of Statistics, University of California, Berkeley (2003)
Saul, L., Jaakkola, T., Jordan, M.: Mean Field Theory for Sigmoid Belief Networks. Journal of Artificial Intelligence Research 4, 61–76 (1996)
Ghahramani, Z., Jordan, M.: Factorial Hidden Markov Models. Machine learning 29, 245–273 (1997)
Xing, E., Jordan, M., Russell, S.: A Generalized Mean Field Algorithm for Variational Inference in Exponential Families. In: 19th Conference on Uncertainty in Artificial Intelligence, pp. 583–591. Morgan Kaufmann, San Francisco (2003)
Yu, S.X., Shi, J.: Multiclass Spectral Clustering. In: 9th IEEE International Conference on Computer Vision, pp. 313–319. IEEE Computer Society, Washington (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chen, Y., Liao, S. (2008). Cluster Selection Based on Coupling for Gaussian Mean Fields. In: Sun, F., Zhang, J., Tan, Y., Cao, J., Yu, W. (eds) Advances in Neural Networks - ISNN 2008. ISNN 2008. Lecture Notes in Computer Science, vol 5263. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87732-5_49
Download citation
DOI: https://doi.org/10.1007/978-3-540-87732-5_49
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87731-8
Online ISBN: 978-3-540-87732-5
eBook Packages: Computer ScienceComputer Science (R0)