Skip to main content
SpringerLink
Log in
Book cover

Non-negative Matrix Factorization Techniques pp 1–48Cite as

From Binary NMF to Variational Bayes NMF: A Probabilistic Approach

  • Chapter
  • First Online:

  • 1914 Accesses

Part of the book series: Signals and Communication Technology ((SCT))

Abstract

A survey of our recent work on probabilistic NMF is provided. All variants discussed here are illustrated by their application to the analysis of failure patterns emerging from manufacturing and processing silicon wafers. It starts with binNMF, a variant developed to apply NMF to binary data sets. The latter are modeled as a probabilistic superposition of a finite number of intrinsic continuous-valued failure patterns characteristic for the manufacturing process. We further discuss related theoretical work on a semi-non-negative matrix factorization based on the logistic function, which we called logistic NMF. While addressing uniqueness issues, we propose a Bayesian Optimality Criterion for NMF and a determinant criterion to geometrically constrain the solutions of NMF problems, leading to detNMF. This approach also provides an intuitive explanation for the often used multilayer approach. Finally, we present a Variational Bayes NMF (VBNMF) algorithm which represents a generalization of the famous Lee–Seung method. We also demonstrate its ability to estimate the intrinsic dimension (model order) of the NMF method.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. R. Schachtner, G. Pöppel, E.W. Lang, IEEE Trans. Circuits Syst. I 57(7), 1439 (2010)

    Article  MathSciNet  Google Scholar 

  2. A.M. Tomé, R. Schachtner, V. Vigneron, C.G. Puntonet, E.W. Lang, Multidimensional Systems and Signal Processing (2013), pp. 1–19. doi:10.1007/s11045-013-0240-9

    Google Scholar 

  3. R. Schachtner, G. Pöppel, E.W. Lang, Digit. Signal Process. 21(4), 528 (2011)

    Article  Google Scholar 

  4. R. Schachtner, G. Pöppel, A. Tomé, C. Puntonet, E.W. Lang, Neurocomputing 138, 142 (2014)

    Google Scholar 

  5. A. Cichocki, arXiv:1403.2048v4 [cs.ET] 24 Aug 2014 (2014), pp. 1–30

  6. P. Paatero, U. Tapper, Environmetrics 5(2), 111 (1994)

    Article  Google Scholar 

  7. P. Paatero, Chemom. Intell. Lab. Syst. 37, 23 (1997)

    Article  Google Scholar 

  8. D.D. Lee, H.S. Seung, Nature 401(6755), 788 (1999)

    Article  Google Scholar 

  9. A. Cichocki, S. Amari, R. Zdunek, A.H. Phan, Non-negative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation (Wiley-Blackwell, Oxford, 2009)

    Book  Google Scholar 

  10. R. Schachtner, G. Pöppel, E.W. Lang, in \(32{nd}\) Annual Meeting of the German Classification Society (GfKl) (2008), pp. 755–764

    Google Scholar 

  11. R. Schachtner, G. Pöppel, A.M. Tomé, E.W. Lang, in Proceedings of 8th International Conference on Independent Component Analysis and Signal Separation, ICA 2009, Paraty, Brazil, 15–18 March 2009, pp. 106–113. doi:10.1007/978-3-642-00599-2_14

    Google Scholar 

  12. R. Schachtner, G. Pöppel, E.W. Lang, in Proceedings of the 2nd International Workshop on Cognitive Information Processing on Elba Island, CIP2010 (2010), pp. 57–62

    Google Scholar 

  13. R. Schachtner, G. Pöppel, A.M. Tomé, E.W. Lang, Pattern Recognit. Lett. 45, 251 (2014). doi:10.1016/j.patrec.2014.04.013

    Article  Google Scholar 

  14. A. Cichocki, R. Zdunek, Advances in Neural Networks (ISNN 2007). Lecture Notes in Computer Science, vol. 4493 (Springer, Berlin, 2007), pp. 793–802

    Google Scholar 

  15. D.D. Lee, H.S. Seung, NIPS 13, 556–562 (2001)

    Google Scholar 

  16. A.I. Schein, L.K. Saul, L.H. Ungar, in Proceedings of the 9th International Workshop on Artificial Intelligence and Statistics (2003), pp. 1–8

    Google Scholar 

  17. S. Lee, J.Z. Huang, J. Hu, Ann. Appl. Stat. 4(3), 1579 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  18. E. Meeds, Z. Ghahramani, R.M. Neal, S.T. Roweis, Bernoulli 19(8), 977 (2007)

    Google Scholar 

  19. E. Lang, R. Schachtner, D. Lutter, D. Herold, A. Kodewitz, F. Blöchl, F.J. Theis, I.R. Keck, J.G. Saez, P.G. Vilda, A.M. Tomé, New advances in biomedical signal processing, in Exploratory Matrix Factorization Techniques for Large Scale Biomedical Data Sets, ed. by J.M. Górriz-Sáez, E.W. Lang, J. Ramírez (Bentham Science Publishers, 2011), pp. 26–47. doi:10.2174/97816080521891110101

    Google Scholar 

  20. A. Cichocki, R. Zdunek, S. Amari, in 2006 IEEE International Conference on Acoustics Speech and Signal Processing, ICASSP 2006, Toulouse, France, 14–19 May 2006 (2006), pp. 621–624. doi:10.1109/ICASSP.2006.1661352

  21. M.E. Tipping, in Proceedings of Advances in Neural Information Processing Systems II (NIPS 1998) (1999), pp. 592–598

    Google Scholar 

  22. A. Kabán, E. Bingham, T. Hirsimäki, in Proceedings of 4th SIAM International Conference on Data Mining (2004), pp. 462–466

    Google Scholar 

  23. M. Collins, S. Dasgupta, R.E. Schapire, in NIPS 11 (2001), pp. 592–598

    Google Scholar 

  24. E. Bingham, A. Kaban, M. Fortelius, Pattern Anal. Appl. 12(1), 55 (2009)

    Article  MathSciNet  Google Scholar 

  25. K.H. Knuth, in Proceedings of the 13th European Signal Processing Conference (EUSIPCO 2005) (2005), pp. 1–8. arXiv:1311.3001 [stat.ML]

  26. D.J.C. MacKay, Information Theory, Inference, and Learning Algorithm (Cambridge University Press, Cambridge, 2003). http://www.inference.phy.cam.ac.uk/mackay/itila/

  27. A. Cichocki, R. Zdunek, S. Amari, Csiszar’s Divergences for Non-Negative Matrix Factorization, Family of New Algorithms (Springer, Berlin, 2006), pp. 32–39

    Google Scholar 

  28. I. Dhillon, S. Sra, in Proceedings of Neural Information Processing Systems (NIPS) (2005)

    Google Scholar 

  29. S. Sra, I.S. Dhillon, Nonnegative matrix approximation: algorithms and applications. Technical report, Computer Sciences, University of Texas, Technical Report, # TR-06-27 (2006)

    Google Scholar 

  30. A. Cichocki, R. Zdunek, S. Amari, IEEE Signal Process. Mag. 142, 142 (2008)

    Article  Google Scholar 

  31. P.K. Hopke, in EPA Workshop Proceedings, Materials from the Work shop on UNMIX and PMF as Applied to PM2.5. (2000). http://www.epa.gov/ttnamti1/files/ambient/pm25/workshop/laymen.pdf

  32. P. Sajda, S. Du, T. Brown, L. Parra, R. Stoyanova, in 4th International Symposium on Independent Component Analysis and Blind Signal Separation (2003), pp. 71–76

    Google Scholar 

  33. P. Sajda, S. Du, T.R. Brown, R. Stoyanova, D.C. Shungu, X. Mao, L.C. Parra, IEEE Trans. Med. Imaging 23(12), 1453 (2004)

    Article  Google Scholar 

  34. L. Miao, H. Qi, IEEE Trans. Geosci. Remote Sens. 45(3), 765 (2007)

    Article  Google Scholar 

  35. M.D. Craig, IEEE Trans. Geosci. Remote Sens. 32(3), 542 (1994)

    Article  Google Scholar 

  36. C.M. Bishop, Neural Networks for Pattern Recognition (Oxford University Press, Oxford, 1996)

    MATH  Google Scholar 

  37. P. Hoyer, in Proceedings of the IEEE Workshop on Neural Networks for Signal Processing (2002), pp. 557–565

    Google Scholar 

  38. M.N. Schmidt, H. Laurberg, Comput. Intell. Neurosci. 1 (2008). doi:10.1155/2008/361705

    Google Scholar 

  39. A.T. Cemgil, Comput. Intell. Neurosci. 1 (2009). doi:10.1155/2009/785152

    Google Scholar 

  40. C. Févotte, A.T. Cemgil, in Proceedings of 17th European Signal Processing Conference (EUSIPCO’09) (2009), pp. 1–5

    Google Scholar 

  41. S. Moussaoui, D. Brie, O. Caspary, A. Mohammad-Djafari, in Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (2004), pp. 485–48

    Google Scholar 

  42. T.O. Virtanen, A.T. Cemgil, S.J. Godsill, in Proceedings of IEEE ICASSP (2008), pp. 1–4

    Google Scholar 

  43. C.J. Lin, IEEE Trans. Neural Netw. 18(6), 1589 (2007)

    Article  Google Scholar 

  44. M. Arngren, M.N. Schmidt, J. Larsen, J. Signal Process. Syst. 65(3), 479 (2010)

    Article  Google Scholar 

  45. D. Zhou, H.Y. Gao, Y.J. Zhang, Adv. Mater. Res. 651, 858 (2013)

    Article  Google Scholar 

  46. K. Stadlthanner, F. Theis, C. Puntonet, J.M. Górriz, A.M. Tomé, E.W. Lang, in ISBMDA. LNCS (LNBI), vol. 3745 (Springer, Heidelberg, 2005), pp. 137–148

    Google Scholar 

  47. A. Cichocki, R. Zdunek, Int. J. Neural Syst. 17(6), 431 (2007)

    Article  Google Scholar 

  48. D.J.C. MacKay, Ensemble learning and evidence maximization. Technical report, Cavendish Laboratory, University of Cambridge (1995)

    Google Scholar 

  49. C.M. Bishop, in Advances in Neural Information Processing Systems NIPS (1999), pp. 382–388

    Google Scholar 

  50. M.N. Schmidt, O. Winther, L.K. Hanse, in International Conference on Independent Component Analysis and Signal Separation. Lecture Notes in Computer Science (LNCS), vol. 5441 (Springer, New York, 2009), pp. 540–547

    Google Scholar 

  51. M. Zhong, M. Girolami, J. Mach. Learn. Res. 5, 663 (2009)

    Google Scholar 

  52. M.N. Schmidt, M. Mørup, in European Signal Processing Conference (EUSIPCO) (2010)

    Google Scholar 

  53. M.I. Jordan, Z. Ghahramani, T.S. Jaakkola, L.K. Saul, Mach. Learn. 37, 183 (1998)

    Article  Google Scholar 

  54. C.M. Bishop, in Proceedings 9-th International Conference on Artificial Neural Networks, ICANN (1999), pp. 509–514

    Google Scholar 

  55. H. Attias, Advances in Neural Information Processing Systems, NIPS 12 (MIT Press, Cambridge, 2000)

    Google Scholar 

  56. Z. Ghahramani, M.J. Beal, Advances in Neural Information Processing Systems NIPS 13 (MIT Press, Cambridge, 2001), pp. 507–513

    Google Scholar 

  57. M. Harva, A. Kabán, Signal Process. 87(3), 509 (2007)

    Article  MATH  Google Scholar 

  58. A. Kabán, E. Bingham, Neurocomputing 71(10–12), 2291 (2008)

    Article  Google Scholar 

  59. V.Y.F. Tan, C. Fevotte, in Proceedings of Workshop on Signal Processing with Adaptative Sparse Structured Representations (SPARS’09) (2009), pp. 1–5

    Google Scholar 

  60. V.Y.F. Tan, C. Fevotte, IEEE Trans. Pattern Anal. Mach. Intell. 35(7), 1592 (2013)

    Article  Google Scholar 

  61. M.J. Beal, Z. Ghahramani, Bayesian Anal. 1(4), 793 (2006)

    Article  MathSciNet  Google Scholar 

  62. J.L.W.V. Jensen, Acta Math. 30, 175 (1906)

    Article  MATH  MathSciNet  Google Scholar 

  63. Z. Ghahramani, Advanced Lectures on Machine Learning (2004) pp. 72–112

    Google Scholar 

  64. D.J.C. MacKay, Neural Comput. 4(3), 415 (1992)

    Article  Google Scholar 

  65. D.J.C. MacKay, Maximum Entropy and Bayesian Methods (Kluwer Academic Publishers, Boston, 1996), pp. 43–60

    Book  Google Scholar 

Download references

Acknowledgments

Support by Infineon Technologies AG and the DAAD (PPP Luso-Alem\(\tilde{a}\), PPP Hispano-Alemana) is gratefully acknowledged.

Author information

Authors and Affiliations

  1. CIML/Biophysics, University of Regensburg, 93040, Regensburg, Germany

    R. Schachtner & E. W. Lang

  2. Infineon AG, 93040, Regensburg, Germany

    R. Schachtner & G. Pöppel

  3. DETI/IEETA, Universidade de Aveiro, 3810-193, Aveiro, Portugal

    A. M. Tomé

Authors
  1. R. Schachtner
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. G. Pöppel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. M. Tomé
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. E. W. Lang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to E. W. Lang .

Editor information

Editors and Affiliations

  1. University of Technology Sydney, Sydney, New South Wales, Australia

    Ganesh R. Naik

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Schachtner, R., Pöppel, G., Tomé, A.M., Lang, E.W. (2016). From Binary NMF to Variational Bayes NMF: A Probabilistic Approach. In: Naik, G. (eds) Non-negative Matrix Factorization Techniques. Signals and Communication Technology. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-48331-2_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-48331-2_1

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-48330-5

  • Online ISBN: 978-3-662-48331-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation