Skip to main content
SpringerLink
Log in

Single-Channel Audio Source Separation with NMF: Divergences, Constraints and Algorithms

  • Chapter
  • First Online:
Audio Source Separation

Part of the book series: Signals and Communication Technology ((SCT))

Abstract

Spectral decomposition by nonnegative matrix factorisation (NMF) has become state-of-the-art practice in many audio signal processing tasks, such as source separation, enhancement or transcription. This chapter reviews the fundamentals of NMF-based audio decomposition, in unsupervised and informed settings. We formulate NMF as an optimisation problem and discuss the choice of the measure of fit. We present the standard majorisation-minimisation strategy to address optimisation for NMF with the common \(\beta \)-divergence, a family of measures of fit that takes the quadratic cost, the generalised Kullback-Leibler divergence and the Itakura-Saito divergence as special cases. We discuss the reconstruction of time-domain components from the spectral factorisation and present common variants of NMF-based spectral decomposition: supervised and informed settings, regularised versions, temporal models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://bass-db.gforge.inria.fr/fasst/.

References

  1. C.J.C. Burges, Dimension reduction: a guided tour. Found. Trends Mach. Learn. 2(4), 275–365 (2009)

    Article  MATH  Google Scholar 

  2. P. Comon, Independent component analysis, a new concept ? Sig. process. 36(3), 287–314 (1994)

    Article  MATH  Google Scholar 

  3. B.A. Olshausen, D.J. Field, Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381(6583), 607–609 (1996)

    Article  Google Scholar 

  4. M. Aharon, M. Elad, A. Bruckstein, K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Sig. Process. 54(11), 4311–4322 (2006)

    Article  MATH  Google Scholar 

  5. P. Paatero, U. Tapper, Positive matrix factorization: a non-negative factor model with optimal utilization of error estimates of data values. Environmetrics 5, 111–126 (1994)

    Article  Google Scholar 

  6. D.D. Lee, H.S. Seung, Learning the parts of objects with nonnegative matrix factorization. Nature 401, 788–791 (1999)

    Article  MATH  Google Scholar 

  7. T. Hofmann, Probabilistic latent semantic indexing, in Proceedings of the 22nd International Conference on Research and Development in Information Retrieval (SIGIR) (1999)

    Google Scholar 

  8. Y. Koren, R. Bell, C. Volinsky, Matrix factorization techniques for recommender systems. Computers 42(8), 30–37 (2009)

    Article  Google Scholar 

  9. N. Dobigeon, J.-Y. Tourneret, C. Richard, J.C.M. Bermudez, S. McLaughlin, A.O. Hero, Nonlinear unmixing of hyperspectral images: models and algorithms. IEEE Sig. Proccess. Mag. 31(1), 89–94 (2014)

    Google Scholar 

  10. P. Smaragdis, J.C. Brown, Non-negative matrix factorization for polyphonic music transcription, in Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) (2003)

    Google Scholar 

  11. C. Févotte, N. Bertin, J.-L. Durrieu, Nonnegative matrix factorization with the Itakura-Saito divergence. With application to music analysis. Neural Comput. 21(3), 793–830 (2009)

    Article  MATH  Google Scholar 

  12. A. Cichocki, H. Lee, Y.-D. Kim, S. Choi, Non-negative matrix factorization with \(\alpha \)-divergence. Pattern Recognit. Lett. 29(9), 1433–1440 (2008)

    Article  Google Scholar 

  13. A. Cichocki, R. Zdunek, S. Amari, Csiszar’s divergences for non-negative matrix factorization: family of new algorithms, in Proceedings of International Conference on Independent Component Analysis and Blind Signal Separation (ICA), Charleston SC, USA (2006), pp. 32–39

    Google Scholar 

  14. R. Kompass, A generalized divergence measure for nonnegative matrix factorization. Neural Comput. 19(3), 780–791 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  15. C. Févotte, J. Idier, Algorithms for nonnegative matrix factorization with the beta-divergence. Neural Comput. 23(9), 2421–2456 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  16. I.S. Dhillon, S. Sra, Generalized nonnegative matrix approximations with Bregman divergences, in Advances in Neural Information Processing Systems (NIPS) (2005)

    Google Scholar 

  17. A. Basu, I.R. Harris, N.L. Hjort, M.C. Jones, Robust and efficient estimation by minimising a density power divergence. Biometrika 85(3), 549–559 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  18. S. Eguchi, Y. Kano, Robustifying maximum likelihood estimation, Institute of Statistical Mathematics, Technical report, June 2001, research Memo. 802

    Google Scholar 

  19. D. FitzGerald, M. Cranitch, E. Coyle, On the use of the beta divergence for musical source separation, in Proceedings of the Irish Signals and Systems Conference (2009)

    Google Scholar 

  20. R. Hennequin, R. Badeau, B. David, NMF with time-frequency activations to model non stationary audio events, in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2010), pp. 445–448

    Google Scholar 

  21. E. Vincent, N. Bertin, R. Badeau, Adaptive harmonic spectral decomposition for multiple pitch estimation. IEEE Trans. Audio Speech Lang. Process. 18, 528–537 (2010)

    Article  Google Scholar 

  22. V.Y.F. Tan, C. Févotte, Automatic relevance determination in nonnegative matrix factorization with the beta-divergence. IEEE Trans. Pattern Anal. Mach. Intell. 35(7), 1592–1605 (2013)

    Article  Google Scholar 

  23. C. Févotte, A.T. Cemgil, Nonnegative matrix factorisations as probabilistic inference in composite models, in Proceedings of the 17th European Signal Processing Conference (EUSIPCO), Glasgow, Scotland (2009), pp. 1913–1917

    Google Scholar 

  24. J.F. Canny, GaP: a factor model for discrete data, in Proceedings of the ACM International Conference on Research and Development of Information Retrieval (SIGIR) (2004), pp. 122–129

    Google Scholar 

  25. A.T. Cemgil, Bayesian inference for nonnegative matrix factorisation models. Comput. Intell. Neurosci. 2009,17 (2009). https://doi.org/10.1155/2009/785152. Article ID 785152

  26. P. Smaragdis, B. Raj, M.V. Shashanka, A probabilistic latent variable model for acoustic modeling, in NIPS Workshop on Advances in Models for Acoustic Processing (2006)

    Google Scholar 

  27. T. Virtanen, Monaural sound source separation by non-negative matrix factorization with temporal continuity and sparseness criteria. IEEE Trans. Audio Speech Lang. Process. 15(3), 1066–1074 (2007)

    Article  Google Scholar 

  28. B. King, C. Févotte, P. Smaragdis, Optimal cost function and magnitude power for NMF-based speech separation and music interpolation, in Proceedings of the IEEE International Workshop on Machine Learning for Signal Processing (MLSP), Santander, Spain (2012)

    Google Scholar 

  29. D.R. Hunter, K. Lange, A tutorial on MM algorithms. Am. Stat. 58, 30–37 (2004)

    Article  MathSciNet  Google Scholar 

  30. P. Smaragdis, C. Févotte, G. Mysore, N. Mohammadiha, M. Hoffman, Static and dynamic source separation using nonnegative factorizations: a unified view. IEEE Sig. Process. Mag. 31(3), 66–75 (2014)

    Article  Google Scholar 

  31. T. Virtanen, Sound source separation using sparse coding with temporal continuity objective, in Proceedings of the International Computer Music Conference (ICMC) (2003), pp. 231–234

    Google Scholar 

  32. S. Vembu, S. Baumann, Separation of vocals from polyphonic audio recordings, in Proceedings of the International Conference on Music Information Retrieval (ISMIR) (2005), pp. 337–344

    Google Scholar 

  33. E. Vincent, N. Bertin, R. Gribonval, F. Bimbot, From blind to guided audio source separation: how models and side information can improve the separation of sound. IEEE Sig. Process. Mag. 31(3), 107–115 (2014)

    Article  Google Scholar 

  34. E. Vincent, X. Rodet, Underdetermined source separation with structured source priors, in Proceedings of the International Conference on Independent Component Analysis and Blind Source Separation (ICA) (2004), pp. 327–334

    Google Scholar 

  35. G.J. Mysore, P. Smaragdis, A non-negative approach to semi- supervised separation of speech from noise with the use of temporal dynamics, in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) (2011), pp. 17–20

    Google Scholar 

  36. P. Smaragdis, Convolutive speech bases and their application to supervised speech separation. IEEE Trans. Audio Speech Lang. Process. 15(1), 1–12 (2007)

    Article  Google Scholar 

  37. P. Smaragdis, M. Shashanka, B. Raj, A sparse non-parametric approach for single channel separation of known sounds, in Proceedings of the Neural Information Processing Systems (NIPS) (2009), pp. 1705–1713

    Google Scholar 

  38. J.F. Gemmeke, T. Virtanen, A. Hurmalainen, Exemplar-based sparse representations for noise robust automatic speech recognition. IEEE Trans. Audio Speech Lang. Process. 19(7), 2067–2080 (2011)

    Article  Google Scholar 

  39. T. Virtanen, J. Gemmeke, B. Raj, Active-set Newton algorithm for overcomplete non-negative representations of audio. IEEE Trans. Audio Speech Lang. Process. 21(11), 2277–2289 (2013)

    Article  Google Scholar 

  40. P.D. O’Grady, B.A. Pearlmutter, Discovering speech phones using convolutive non-negative matrix factorisation with a sparseness constraint. Neurocomputing 72(1–3), 88–101 (2008)

    Article  Google Scholar 

  41. W. Wang, A. Cichocki, J.A. Chambers, A multiplicative algorithm for convolutive non-negative matrix factorization based on squared Euclidean distance. IEEE Trans. Sig. Process. 57(7), 2858–2864 (2009)

    Google Scholar 

  42. J.-L. Durrieu, G. Richard, B. David, C. Févotte, Source/filter model for unsupervised main melody extraction from polyphonic audio signals. IEEE Trans. Audio Speech Lang. Process. 18(3), 564–575 (2010)

    Article  Google Scholar 

  43. A. Ozerov, E. Vincent, F. Bimbot, A general flexible framework for the handling of prior information in audio source separation. IEEE Trans. Audio Speech Lang. Process. 20(4), 1118–1133 (2012)

    Article  Google Scholar 

  44. D. FitzGerald, M. Cranitch, E. Coyle, Extended nonnegative tensor factorisation models for musical sound source separation. Comput. Intell. Neurosci. 2008 (2008). Article ID 872425

    Google Scholar 

  45. T. Virtanen, Monaural sound source separation by nonnegative matrix factorization with temporal continuity and sparseness criteria. IEEE Trans. Audio Speech Lang. Process. 15(3), 1066–1074 (2007)

    Article  Google Scholar 

  46. J. Eggert, E. Körner, Sparse coding and NMF, in Proceedings of the IEEE International Joint Conference on Neural Networks (2004), pp. 2529–2533

    Google Scholar 

  47. J. Le Roux, F.J. Weninger, J.R. Hershey, Sparse NMF–half-baked or well done? Mitsubishi Electric Research Laboratories (MERL), Technical report TR2015-023, 2015

    Google Scholar 

  48. C. Joder, F. Weninger, D. Virette, B. Schuller, A comparative study on sparsity penalties for NMF-based speech separation: beyond Lp-norms, in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) (2013), pp. 858–862

    Google Scholar 

  49. Y. Mitsui, D. Kitamura, S. Takamichi, N. Ono, H. Saruwatari, Blind source separation based on independent low-rank matrix analysis with sparse regularization for time-series activity, in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) (2017)

    Google Scholar 

  50. A. Lefèvre, F. Bach, C. Févotte, Itakura-Saito nonnegative matrix factorization with group sparsity, in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) (2011), pp. 21–24

    Google Scholar 

  51. D.L. Sun, G.J. Mysore, Universal speech models for speaker independent single channel source separation, in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2013), pp. 141–145

    Google Scholar 

  52. O. Dikmen, A.T. Cemgil, Gamma Markov random fields for audio source modeling. IEEE Trans. Audio Speech Lang. Process. 18(3), 589–601 (2010)

    Article  Google Scholar 

  53. C. Févotte, J. Le Roux, J.R. Hershey, Non-negative dynamical system with application to speech and audio, in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) (2013), pp. 3158–3162

    Google Scholar 

  54. G. Mysore, M. Sahani, Variational inference in non-negative factorial hidden Markov models for efficient audio source separation, in Proceedings of the International Conference on Machine Learning (ICML) (2012), pp. 1887–1894

    Google Scholar 

  55. A. Ozerov, C. Févotte, R. Blouet, J.-L. Durrieu, Multichannel nonnegative tensor factorization with structured constraints for user-guided audio source separation, in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Prague (2011), pp. 257–260

    Google Scholar 

  56. N.Q.K. Duong, A. Ozerov, L. Chevallier, J. Sirot, An interactive audio source separation framework based on non-negative matrix factorization, in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) (2014), pp. 1567–1571

    Google Scholar 

  57. J.-L. Durrieu, J.-P. Thiran, Musical audio source separation based on user-selected F0 track, in Proceedings of the International Conference on Latent Variable Analysis and Signal Separation (LVA/ICA) (2012), pp. 438–445

    Google Scholar 

  58. S. Ewert, B. Pardo, M. Müller, M.D. Plumbley, Score-informed source separation for musical audio recordings: an overview. IEEE Sig. Process. Mag. 31(3), 116–124 (2014)

    Article  Google Scholar 

  59. P. Smaragdis, G.J. Mysore, Separation by humming: user-guided sound extraction from monophonic mixtures, in Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) (2009), pp. 69–72

    Google Scholar 

  60. L. Le Magoarou, A. Ozerov, N.Q.K. Duong, Text-informed audio source separation using nonnegative matrix partial co-factorization, in Proceedings of the IEEE International Workshop on Machine Learning for Signal Processing (MLSP) (2013), pp. 1–6

    Google Scholar 

  61. N. Souviraà-Labastie, A. Olivero, E. Vincent, F. Bimbot, Multi-channel audio source separation using multiple deformed references. IEEE/ACM Trans. Audio Speech Lang. Process. 23(11), 1775–1787 (2015)

    Article  Google Scholar 

  62. Y.K. Yılmaz, A.T. Cemgil, U. Şimşekli, Generalized coupled tensor factorization, in Advances in Neural Information Processing Systems (NIPS) (2011)

    Google Scholar 

  63. N. Seichepine, S. Essid, C. Févotte, O. Cappé, Soft nonnegative matrix co-factorization. IEEE Trans. Sig. Process. 62(22), 5940–5949 (2014)

    Article  MathSciNet  Google Scholar 

  64. E. Vincent, T. Virtanen, S. Gannot, Audio Source Separation and Speech Enhancement (Wiley, 2017)

    Google Scholar 

Download references

Acknowledgements

Cédric Févotte acknowledges funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme under grant agreement No 681839 (project FACTORY).

Author information

Authors and Affiliations

  1. CNRS & IRIT, Toulouse, France

    Cédric Févotte

  2. Inria, 54600, Villers-lès-Nancy, France

    Emmanuel Vincent

  3. Technicolor, Rennes, France

    Alexey Ozerov

Authors
  1. Cédric Févotte
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Emmanuel Vincent
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexey Ozerov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cédric Févotte .

Editor information

Editors and Affiliations

  1. University of Tsukuba, Ibaraki, Japan

    Shoji Makino

Standard Distributions

Standard Distributions

Poisson

$$\begin{aligned} {Po}(x|\lambda ) = \exp (- \lambda ) \, \frac{\lambda ^x}{x!}, \quad x \in \{0, 1, \ldots , \infty \} \end{aligned}$$
(1.50)

Multinomial

$$\begin{aligned} {M}( {\mathbf {x}} |N, {\mathbf {p}})= \frac{N !}{x_1 ! \ldots x_K!} p_1^{x_1} \ldots p_K^{x_K}, \quad x_k \in \{0,\ldots ,N \}, \sum _k x_k = N \end{aligned}$$
(1.51)

Circular complex normal distribution

$$\begin{aligned} {N}_c \left( {x}|\mu , \varSigma \right) = |\pi \, \varSigma |^{-1}\,\exp {- ({x} - \mu )^H\,\varSigma ^{-1}\,({x} - \mu )}, \quad x \in \mathbb {C}^F \end{aligned}$$
(1.52)

Gamma

$$\begin{aligned} {G}(x | \alpha , \beta ) = \frac{\beta ^{\alpha }}{\varGamma (\alpha )}\, x^{\alpha -1}\, \exp (-\beta \, x), \quad x \ge 0 \end{aligned}$$
(1.53)

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Févotte, C., Vincent, E., Ozerov, A. (2018). Single-Channel Audio Source Separation with NMF: Divergences, Constraints and Algorithms. In: Makino, S. (eds) Audio Source Separation. Signals and Communication Technology. Springer, Cham. https://doi.org/10.1007/978-3-319-73031-8_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-73031-8_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-73030-1

  • Online ISBN: 978-3-319-73031-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation