Abstract
Binary Factor Analysis (BFA) aims to discover latent binary structures in high dimensional data. Parameter learning in BFA suffers from exponential complexity and a large number of local optima. Model selection in BFA is therefore difficult. The traditional approach for model selection is implemented in a two phase procedure. On a prefixed range of model scales, maximum likelihood (ML) learning is performed for each candidate scale. After this enumeration, the optimum scale is selected according to some criterion. In contrast, the Bayesian Ying-Yang (BYY) learning starts from a high dimensional model and automatically deducts the dimension during parameter learning. The enumeration overhead in the two phase approach is saved. This paper investigates a subclass of BFA called Orthogonal Binary Factor Analysis (OBFA). A BYY machine for OBFA is constructed. The harmony measure, which serves as the objective function in the BYY harmony learning, is more accurately estimated by recovering a term that was missing in the previous studies on BYY learning based BFA. Comparison with traditional two phase implementations shows good performance of the proposed approach.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Anderson, T.W., Rubin, H.: Statistical inference in factor analysis. In: Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, vol. 5, pp. 111–150 (1956)
Bartholomew, D.J., Knott, M.: Latent variable models and factor analysis. Kendallas Library of Statistics, vol. 7. Oxford University Press, New York (1999)
Heinen, T.: Latent class and discrete latent trait models: Similarities and differences. Sage, Thousand Oaks (1996)
Treier, S., Jackman, S.: Beyond factor analysis: modern tools for social measurement. In: The 2002 Annual Meetings of the Western Political Science Association and the Midwest Political Science Association (2002)
Tapchi, G.: System of Experimental Design: Engineering Methods to Optimize Quality and Minimize Costs. UNIPUB/Kraus International Publications (1987)
Akaike, H.: A new look at the statistical model identification. IEEE Trans. Automatic Control 19(6), 716–723 (1974)
Bozdogan, H.: Model selection and Akaikeas information criterion (AIC): the general theory and its analytical extensions. Psychometrika 52(3), 345–370 (1987)
Schwarz, G.: Estimating the dimension of a model. The Annals of Statistics 6(2), 461–464 (1978)
Rissanen, J.: Modeling by shortest data description. Automatica 14, 465–471 (1978)
Xu, L.: A Unified Learning Scheme: Bayesian-Kullback Ying-Yang Machine. In: Touretzky, D.S., et al. (eds.) Advances in NIPS, vol. 8, pp. 444–450. MIT Press, Cambridge (1996); A preliminary version in Proc. ICONIP 1995, Beijing, pp. 977–988 (1995)
Xu, L.: Advances on BYY harmony learning: information theoretic perspective, generalized projection geometry, and independent factor auto-determination. IEEE Trans. Neural Networks 15(5), 885–902 (2004)
Xu, L.: Bayesian Ying Yang learning. Scholarpedia 2(3), 1809 (2007), http://scholarpedia.org/article/Bayesian_Ying_Yang_Learning
Xu, L.: Bayesian Ying Yang System, Best Harmony Learning, and Gaussian Manifold Based Family. In: Zurada, J.M., et al. (eds.) Computational Intelligence: Research Frontiers. LNCS, vol. 5050, pp. 48–78. Springer, Heidelberg (2008)
An, Y., et al.: A Comparative Investigation on Model Selection in Binary Factor Analysis. Studies in Computational Intelligence 6, 145–160 (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sun, K., Xu, L. (2008). Bayesian Ying-Yang Learning on Orthogonal Binary Factor Analysis. In: Kůrková, V., Neruda, R., Koutník, J. (eds) Artificial Neural Networks - ICANN 2008. ICANN 2008. Lecture Notes in Computer Science, vol 5163. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87536-9_27
Download citation
DOI: https://doi.org/10.1007/978-3-540-87536-9_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87535-2
Online ISBN: 978-3-540-87536-9
eBook Packages: Computer ScienceComputer Science (R0)