Science in China Series F

, Volume 46, Issue 1, pp 31–44 | Cite as

Grading learning for blind source separation



By generalizing the learning rate parameter to a learning rate matrix, this paper proposes a grading learning algorithm for blind source separation. The whole learning process is divided into three stages: initial stage, capturing stage and tracking stage. In different stages, different learning rates are used for each output component, which is determined by its dependency on other output components. It is shown that the grading learning algorithm is equivariant and can keep the separating matrix from becoming singular. Simulations show that the proposed algorithm can achieve faster convergence, better steady-state performance and higher numerical robustness, as compared with the existing algorithms using fixed, time-descending and adaptive learning rates.


blind source separation independent component analysis neural computation adaptive learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hyvarinen, A., Fast and robust fixed-point algorithms for independent component analysis, IEEE Trans. Neural Networks, 1999, 10(3): 626–634.CrossRefGoogle Scholar
  2. 2.
    Giannakopoulos, V., Comparison of adaptive independent component analysis algorithms, Available at Scholar
  3. 3.
    Bell, A. J., Sejnowski, T. J., An information-maximization approach to blind separation and blind deconvolution, Neural Computations, 1995, 7: 1129–1159.CrossRefGoogle Scholar
  4. 4.
    Karhunen, J., Joutsensalo, J., Representation and separation of signals using nonlinear PCA type learning, Neural Networks, 1994, 7: 113–127.CrossRefGoogle Scholar
  5. 5.
    Karhunen, J., Pajunen, P., Oja, E., The nonlinear PCA criterion in blind source separation: Relations with other approaches, Neural Computing, 1998, 22: 5–20.MATHGoogle Scholar
  6. 6.
    Comon, P., Independent component analysis, a new concept? Signal Processing, 1994, 36: 287–314.MATHCrossRefGoogle Scholar
  7. 7.
    Pham, D. T., Blind separation of instantaneous mixtures of sources via an independent component analysis, IEEE Trans. Signal Processing, 1996, 44: 2768–2779.CrossRefGoogle Scholar
  8. 8.
    Cardoso, J. F., Laheld, B., Equivariant adaptive source separation, IEEE Trans. Signal Processing, 1996, 44: 3017–3030.CrossRefGoogle Scholar
  9. 9.
    Yang, H. H., Amari, S., Adaptive on-line learning algorithms for blind separation—maximum entropy and minimum mutual information, Neural Computations, 1997, 9: 1457–1482.CrossRefGoogle Scholar
  10. 10.
    Yang, H. H., Serial updating rule for blind separation derived from the method of scoring, IEEE Trans. Signal Processing, 1999, 47: 2279–2285.CrossRefGoogle Scholar
  11. 11.
    Leen, T. W., Girolami, M., Sejnowski, T. J., Independent component analysis using an extended informax algorithm for mixed sub-Gaussian and super-Gaussian sources, Neural Computations, 1999, 11: 409–433.Google Scholar
  12. 12.
    Choi, S., Cichocki, A., Amari, S., Flexible independent component analysis, Journal of VLSI Signal Processing, 2000, 26: 25–38.MATHCrossRefGoogle Scholar
  13. 13.
    Cruces, S., Cichocki, A., Castedo, L., An iterative inversion approach to blind source separation, IEEE Trans. Neural Networks, 2000, 11: 1423–1437.CrossRefGoogle Scholar
  14. 14.
    Amari, S., Cichocki, A., Yang, H. H., A new learning algorithm for blind signal separation, in Advances in NIPS, 1996, 8: 757–763.Google Scholar
  15. 15.
    Amari, S., Natural gradient works efficiently in learning, Neural Computations, 1998, 10: 251–276.CrossRefGoogle Scholar
  16. 16.
    Cichocki, A., Amari, S., Adachi, M. et al., Self-adaptive neural networks for blind separation of sources, Proc. 1996 International Symp. on Circuits and Systems, vol.2, New York: IEEE Press, May 1996, 157–160.Google Scholar
  17. 17.
    Murata, N., Muller, K. R., Ziehe, A. et al., Adaptive on-line learning in changing environments. Advances in NIPS’9, Cambridge, MA: MIT Press, 1997, 599–605.Google Scholar
  18. 18.
    Douglas, S. C., Cichocki, A., Adaptive step size techniques for decorrelation and blind source separation, Proc. 32nd Asilomar Conf. on Signals, Systems and Computers, Pacific Grove, CA, vol.2, New York: IEEE Press, Nov. 1998, 1191–1195.Google Scholar
  19. 19.
    Jacobs, R. A., Increased rates of convergence through learning rate adaptation, Neural Networks, 1988, 1: 295–307.CrossRefGoogle Scholar
  20. 20.
    Mathews, V. J., Xie, Z., A stochastic gradient adaptive filter with gradient adaptive step size, IEEE Trans. Signal Processing, 1993, 41: 2075–2087.CrossRefGoogle Scholar
  21. 21.
    Zhang, X. D., Bao, Z., Communication Signal Processing (in Chinese), Beijing: Defense Industry Press, 2000, 367–383.Google Scholar
  22. 22.
    Papoulis, A., Probability, Random Variables, and Stochastic Process, 3rd edition, New York: McGraw-Hill, 1991, 190–191.Google Scholar

Copyright information

© Science in China Press 2003

Authors and Affiliations

  1. 1.Key Laboratory for Radar Signal ProcessingXidian UniversityXi’anChina
  2. 2.Department of Automation, State Key Laboratory of Intelligent Technology and SystemsTsinghua UniversityBeijingChina

Personalised recommendations