Skip to main content
SpringerLink
Log in

Blind Source Recovery: A State-Space Formulation

  • Chapter
Stability and Control of Dynamical Systems with Applications

Part of the book series: Control Engineering ((CONTRENGIN))

Abstract

Blind source recovery (BSR) denotes recovery of original sources or signals without any explicit identification of the environments which may include convolution, temporal variation, and even nonlinearity. This chapter provides an overview of a generalized (i.e., nonlinear and time-varying) state-space BSR formulation by the application of stochastic optimization principles to the Kullback-Lieblar divergence as an information-theoretic performance functional. The multivariable optimization technique is used to derive update laws for nonlinear time-varying dynamical systems, which are subsequently specialized to time-invariant and linear systems. Furthermore, the various possible state-space demixing network structures have been exploited to develop learning rules, capable of handling most filtering paradigms—which are conveniently extendible to nonlinear models. Distinct linear state-space algorithms are presented for the minimum phase and nonminimum phase mixing environment models. Illustrative simulation examples are then presented to demonstrate the on-line adaptation capabilities of the developed algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. S. Amari Differential-Geometrical Methods in Statistics Lecture Notes in Statistics 28. Springer-Verlag, New York, 1985.

    Book  Google Scholar 

  2. S. Amari, “Neural learning in structured parameter spaces-Natural Riemannian gradient,” in Neural Information Processing Systems NIPS-96, pp. 127–133, MIT Press, Cambridge, MA, 1996.

    Google Scholar 

  3. S. Amari and J. F. Cardoso, “Blind source separation: Semi-parametric statistical approach,” IEEE Trans. Signal Processing vol. 45, no. 11, pp. 2692–2700, 1997.

    Article  Google Scholar 

  4. S. Amari, S. C. Douglas, A. Cichocki, and H. H. Yang, “Multichannel blind deconvolution and equalization using the natural gradient,” Proc. IEEE Workshop on Signal Processing pp. 101–104, Paris, France, 1997.

    Google Scholar 

  5. S. Amari, T.-P. Chen, and A. Cichocki, “Stability analysis of adaptive blind source separation,” Neural Networks vol. 10, no. 8, pp. 1345–1352, 1997.

    Article  Google Scholar 

  6. S. Amari, “Natural gradient works efficiently in learning,” Neural Computation vol. 10, pp. 251–276,1998.

    Article  Google Scholar 

  7. S. Amari and H. Nagaoka Methods of Information Geometry AMS and Oxford University Press, London, 1999.

    Google Scholar 

  8. P. J. Antsaklis and A. N. Michel Linear Systems McGraw-Hill, New York, 1997.

    Google Scholar 

  9. J. Bell and T. J. Sejnowski, “An information-maximization approach to blind separation and blind deconvolution,” Neural Computation vol. 7, pp. 1129–1159, 1995.

    Article  Google Scholar 

  10. A. Benveniste, M. Goursat and G. Ruget, “Robust identification of a non-minimum phase system: Blind adjustment of a linear equalizer in data communications,” IEEE Trans. Automatic Control vol. 25, pp. 385–399, June 1980.

    Article  MathSciNet  MATH  Google Scholar 

  11. J. F. Cardoso, “Blind signal processing: Statistical principles,” Proc. IEEE vol. 90, no. 8, pp. 2009–20026, Oct. 1998 (Special issue on Blind Identification and Estimation, R.-W. Liu and L. Tong, eds.).

    Article  Google Scholar 

  12. J. F. Cardoso and B. Laheld, “Equivariant adaptive source separation,” IEEE Trans. Signal Processing vol. 44, pp. 3017–3030, 1996.

    Article  Google Scholar 

  13. A. Cichocki, S. Amari, and J. Cao, “Neural network models for blind separation of time delayed and convolved signals,” Japanese IECE Trans. Fundamentals vol. E82-A, pp. 1595–1603, 1997.

    Google Scholar 

  14. A. Cichocki and S. Amari Adaptive Blind Signal and Image Processing John Wiley, New York, 2002.

    Book  Google Scholar 

  15. S. C. Douglas and S. Haykin, “On the relationship between blind deconvolution and blind source separation,” Proc. 31st Asilomar Conference on Signals Systems and Computers vol. 2, pp. 1591–1595, Pacific Grove, CA, 1997.

    Google Scholar 

  16. G. Erten and F. M. Salam, “Voice output extraction by signal separation,” Proc. 1998 IEEE International Symposium on Circuits and Systems vol. 3, pp. 5–8, Monterey, CA, 1998.

    Google Scholar 

  17. G. Erten and F. M. Salam, “Voice extraction by on-line signal separation and recovery,” IEEE Trans. Circuits and Systems II: Analog and Digital Signal Processing vol. 46, no. 7, pp. 915–922, July 1999.

    Article  Google Scholar 

  18. A. B. A. Gharbi and E M. Salam, “Separation of mixed signals in dynamic environments: Formulation and some implementation,” Proc. 37th Midwest Symposium on Circuits and Systems vol. 1, pp. 587–590, Lafayette, LA, 1994.

    Google Scholar 

  19. A. B. A. Gharbi and F. M. Salam, “Implementation and test results of a chip for the separation of mixed signals,” IEEE Trans. Circuits and Systems vol. 42, no. 11, pp. 748–751, Nov. 1995.

    Article  Google Scholar 

  20. A. B. A. Gharbi and F. M. Salam, “Algorithms for blind signal separation and recovery in static and dynamic environments,” Proc. 1997 International Symposium on Circuits and Systems pp. 713–716, Hong Kong, June 1997.

    Google Scholar 

  21. M. Girolami Advances in Independent Component Analysis Springer-Verlag, New York, 2000.

    Book  MATH  Google Scholar 

  22. S. Haykin Adaptive Filter Theory 3rd ed., Prentice-Hall, Upper Saddle River, NJ, 1996.

    Google Scholar 

  23. S. Haykin, Ed. Unsupervised Adaptive Filtering vol. I and II, John Wiley, New York, 2000.

    Google Scholar 

  24. J. Herault and C. Jutten, “Space or time adaptive signal processing by neural network models,” in Neural Networks for Computing AIP Conference Proceedings vol. 151, pp. 206–211, American Institute for Physics, New York, 1986.

    Google Scholar 

  25. A. Hyvarinen and E. Oja, `A fast fixed-point algorithm for independent component analysis,“ Neural Computation vol. 9, pp. 1483–1492, 1997.

    Article  Google Scholar 

  26. A. Hyvarinen, J. Karhunen, and E. Oja Independent Component Analysis John Wiley, New York, 2001.

    Book  Google Scholar 

  27. M. Kendall and A. Stuart The Advanced Theory of Statistics vol. I, Charles Griffin and Co. Ltd., London, 1977.

    MATH  Google Scholar 

  28. H. K. Khalil Nonlinear Systems 3rd ed., Préntice-Hall, Upper Saddle River, NJ, 2002.

    MATH  Google Scholar 

  29. R. Lambert Multichannel Blind Deconvolution: FIR Matrix Algebra and Separation of Multipath Mixtures Ph.D. Thesis, University of Southern California, Department of Electrical Engineering, 1996.

    Google Scholar 

  30. T. W. Lee and T. Sejnowski, “Independent component analysis for sub-Gaussian and super-Gaussian mixtures,” Proc. 4th Joint Symposium on Neural Computation vol. 7, pp. 132–140, Institute for Neural Computation, 1997.

    Google Scholar 

  31. T. W. Lee, A. J. Bell, and R. Orglmeister, “Blind source separation of real-world signals,” Proc. IEEE Conference on Neural Networks pp. 2129–2135, Houston, TX, 1997.

    Google Scholar 

  32. T. W. Lee, M. Girolami, A. Bell, and T. J. Sejnowski, “A unifying information-theoretic framework for independent component analysis,” Int. J. Mathematical and Computer Modeling vol. 38, pp. 1–21,2000.

    MathSciNet  Google Scholar 

  33. F. L. Lewis and V. L. Syrmos Optimal Control 2nd ed., Wiley, New York, 1995.

    Google Scholar 

  34. A. N. Michel and D. Liu Qualitative Analysis and Synthesis of Recurrent Neural Networks Marcel Dekker, New York, 2002.

    MATH  Google Scholar 

  35. J. Nicholls, P. A. Fuchs, A. R. Martin, and B. G. Wallace From Neuron to Brain 4th ed., Sinauer Associates Inc., Sunderland, MA, 2001.

    Google Scholar 

  36. F M. Salam,“An adaptive network for blind separation of independent signals,” Proc. IEEE International Symposium on Circuits and Systems vol. 1, pp. 431–434, Chicago, IL, May 1993.

    Google Scholar 

  37. F. M. Salam and G. Erten, “Blind signal separation and recovery in dynamic environments,” Proc.3rd IEEE Workshop on Nonlinear Signal and Image Processing Mackinac Island, MI, Sept. 1997http://www.ecn.purdue.edu/NSIP/ta34.ps.

    Google Scholar 

  38. F. M. Salam and G. Erten, “Exact entropy series representation for blind source separation,” Proc. IEEE Conference on Systems Man and Cyberneticsvol. 1, pp. 553–558, 1999.

    Google Scholar 

  39. F. M. Salam and G. Erten, “The state-space framework for blind dynamic signal extraction and recovery,” Proc. 1999 IEEE International Symposium on Circuits and Systems vol. 5, pp. 66–69, Orlando, FL, 1999.

    Google Scholar 

  40. F M. Salam, A. B. A. Gharbi, and G. Erten, “Formulation and algorithms for blind signal recovery,” Proc. 40th Midwest Symposium on Circuits and Systems, vol. 2, pp. 1233–1236, Sacramento, CA, 1998.

    Google Scholar 

  41. F. M. Salam, G. Erten, and K. Waheed, “Blind source recovery: Algorithms for static and dynamic environments,” Proc. of INNS-IEEE International Joint Conference on Neural Networks vol. 2, pp. 902–907, Washington, DC, 2001.

    Google Scholar 

  42. F. M. Salam and K. Waheed, “State-space feedforward and feedback structures for blind source recovery,” Proc. 3rd International Conference on Independent Component Analysis and Blind Signal Separation pp. 248–253, San Diego, CA, 2001.

    Google Scholar 

  43. K. Torkkola, “Blind separation of convolved sources based on information maximization,” Proc. IEEE Workshop on Neural Networks for Signal Processing pp. 423–432, Kyoto, Japan, 1996.

    Google Scholar 

  44. K. Waheed and F. M. Salam, “Blind source recovery: Some implementation and performance issues,” Proc. 44th IEEE Midwest Symposium on Circuits and Systems vol. 2, pp. 694–697, Dayton, OH, 2001.

    Google Scholar 

  45. K. Waheed and F. M. Salam, “State-space blind source recovery for mixtures of multiple source distributions,” Proc. IEEE International Symposium on Circuits and Systems pp. 197–200, Scottsdale, AZ, May 2002

    Google Scholar 

  46. K. Waheed and F. M. Salam, “Blind source recovery using an adaptive generalized Gaussian score function,” Proc. 45th IEEE Midwest Symposium on Circuits and Systems vol. 2, pp. 418–421, Tulsa, OK, Aug. 2002.

    Google Scholar 

  47. K. Waheed and F. M. Salam, “A data-derived quadratic independence measure for adaptive blind source recovery in practical applications,” Proc. 45th IEEE Midwest Symposium on Circuits and Systems vol. 3, pp. 473–476, Tulsa, OK, Aug. 2002.

    Google Scholar 

  48. K. Waheed and F. M. Salam, “Cascaded structures for blind source recovery,” Proc. 45th IEEE Midwest Symposium on Circuits and Systems vol. 3, pp. 656–659, Tulsa, OK, Aug. 2002.

    Google Scholar 

  49. K. Waheed and F. M. Salam, “State-space blind source recovery of non-minimum phase environments,” Proc. 45th IEEE Midwest Symposium on Circuits and Systems vol. 2, pp. 422–425, Tulsa, OK, Aug. 2002.

    Google Scholar 

  50. K. Waheed and E M. Salam, “New hyperbolic source density models for blind source recovery score functions,” IEEE International Symposium on Circuits and Systems Bangkok, Thailand, May 2003, forthcoming.

    Google Scholar 

  51. K. Waheed and E M. Salam,“Algebraic overcomplete independent component analysis,” 4th International Symposium on Independent Component Analysis and Blind Source Separation Nara, Japan, Apr. 2003, forthcoming.

    Google Scholar 

  52. H. H. Yang, S. Amari, and A. Cichocki, “Information back-propagation for blind separation of sources from non-linear mixtures,” Proc. IEEE Conference on Neural Networks pp. 2141–2146, Houston, TX, 1997.

    Google Scholar 

  53. L. Q. Zhang, A. Cichocki, and S. Amari, “Multichannel blind deconvolution of nonminimum phase systems using information backpropagation,” Proc. 5th International Conference on Neural Information Processing pp. 210–216, Perth, Australia, Nov. 1999.

    Google Scholar 

  54. L. Q. Zhang, A. Cichocki, and S. Amari, “Kalman filter and state-space approach to multichannel blind deconvolution,” Proc. IEEE Workshop on Neural Networks for Signal Processing pp. 425–434, Sydney, Australia, Dec. 2000.

    Google Scholar 

  55. L. Q. Zhang, S. Amari, and A. Cichocki, “Semiparametric approach to multichannel blind deconvolution of non-minimum phase systems,” Advances in Neural Information Processing Systems vol. 12, pp. 363–369, MIT Press, Cambridge, MA, 2000.

    Google Scholar 

  56. L. Q. Zhang, and A. Cichocki, “Blind deconvolution of dynamical systems: A state space approach,” J. Signal Processing vol. 4, no. 2 pp. 111–130, Mar. 2000.

    Google Scholar 

  57. Website http://www.egr.msu.edu/bsr.

Download references

Authors
  1. Khurram Waheed
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fathi M. Salem
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Illinois at Chicago, Chicago, Illinois, 60607, USA

    Derong Liu

  2. Department of Electrical Engineering, University of Notre Dame, Notre Dame, IN, 46556, USA

    Panos J. Antsaklis

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer Science+Business Media New York

About this chapter

Cite this chapter

Waheed, K., Salem, F.M. (2003). Blind Source Recovery: A State-Space Formulation. In: Liu, D., Antsaklis, P.J. (eds) Stability and Control of Dynamical Systems with Applications. Control Engineering. Birkhäuser, Boston, MA. https://doi.org/10.1007/978-1-4612-0037-6_9

Download citation

  • DOI: https://doi.org/10.1007/978-1-4612-0037-6_9

  • Publisher Name: Birkhäuser, Boston, MA

  • Print ISBN: 978-1-4612-6583-2

  • Online ISBN: 978-1-4612-0037-6

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation