Abstract
This paper addresses the problem of transforming a set of patterns into new patterns whose components are mutually statistically independent. This is a problem that, depending on the specific setting, is often designated as factorial coding, independent components analysis, source separation or nonlinear principal components analysis. The reasons for the growing interest on this problem are briefly examined. Some of the most important methods that have been proposed to solve the problem are overviewed, A new class of objective functions for solving the problem is presented. These new objective functions have the advantage of being continuous and differentiable even when they are computed from the empirical distribution of the training data. Some examples of the use of these objective functions are given.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
H. B. Barlow, Unsupervised Learning, Neural Computation, 1 (1989), pp. 295–311.
C. Jutten and J. Hérault, Blind Separation of Sources, Part I: An Adaptive Algorithm Based on a Neuromimetic Architecture, Signal Processing, 24 (1991), pp. 1–10.
P. Comon, Independent Component Analysis, A New Concept?Signal Processing, 36 (1994), pp. 287–314.
L. Wang, J. Karhunen and E. Qja, A Bigradient Optimization Approach for Robust PCA, MCA and Source Separation, in Proc. ICNN’95 (Perth, Australia, 1995).
J. Karhunen, L. Wang and R. Vigario, Nonlinear PC A Type Approaches for Source Separation and Independent Component Analysis, in Proc. ICNN’95, (Perth, Australia, 1995).
A. N. Redlich, Supervised Factorial Learning Neural Computation, 5 (1993), pp. 750–766.
G. Deco and W. Brauer, Nonlinear Higher-Order Statistical Decorrelation by Volume-Conserving Neural Architectures, Neural Networks, 8 (1995), pp. 525–535.
G. E. Hinton and R. S. Zemel, Autoencoders, Minimum Description Length and Helmholtz Free Energy, in Advances in Neural Information Processing Systems 6, eds. J. Cowan, G. Tesauro and I. Alspector (Morgan Kaufman, San Mateo, CA, 1994), pp. 3–10.
J. Rissanen, Stochastic Complexity in Statistical Inquiry (World Scientific Publishing Co., Singapore, 1989).
S. Amari, A. Cichocki and H. H. Yang, A New Learning Algorithm for Blind Signal Separation, Neural Information Processing Systems: Natural and Synthetic (MIT Press, 1996) in print.
A. J. Bell and T. J. Sejnowski, An Information-Maximization Approach to Blind Separation and Blind Deconvolution, Neural Computation, 7 (1995), pp. 1129–1159.
J. Schmidhuber, Learning Factorial Codes by Predictability Minimization, Neural Computation, 4 (1992), pp. 863–879.
G. Marques and L. B. Almeida, An Objective Function for Independence, in Proc. ICNN’96, (Washington DC, 1996), pp. 453–457, in print.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1997 Springer-Verlag London Limited
About this paper
Cite this paper
Almeida, L.B., Marques, G.C. (1997). A Class of Cost Functions for Independence. In: Marinaro, M., Tagliaferri, R. (eds) Neural Nets WIRN VIETRI-96. Perspectives in Neural Computing. Springer, London. https://doi.org/10.1007/978-1-4471-0951-8_1
Download citation
DOI: https://doi.org/10.1007/978-1-4471-0951-8_1
Publisher Name: Springer, London
Print ISBN: 978-1-4471-1240-2
Online ISBN: 978-1-4471-0951-8
eBook Packages: Springer Book Archive