Nonnegative Matrix Factorization with Fixed L2-Norm Constraint

Abstract

Nonnegative matrix factorization (NMF) is a very attractive scheme in learning data representation, and constrained NMF further improves its ability. In this paper, we focus on the L2-norm constraint due to its wide applications in face recognition, hyperspectral unmixing, and so on. A new algorithm of NMF with fixed L2-norm constraint is proposed by using the Lagrange multiplier scheme. In our method, we derive the involved Lagrange multiplier and learning rate which are hard to tune. As a result, our method can preserve the constraint exactly during the iteration. Simulations in both computer-generated data and real-world data show the performance of our algorithm.

Appendix

1.1 The Proof of Lemma 2

Proof

Part (1): Since $\mathbf{I }+\frac{\tau }{2}\mathbf{A }=\mathbf{I }+\frac{\tau }{2}\mathbf{MN }^\mathrm{T}$, we apply the SMW formula:

$$\begin{aligned} \begin{aligned} (\mathbf{B }+\alpha \mathbf{MN }^\mathrm{T})^{-1}=\mathbf{B }^{-1}-\alpha \mathbf{B }^{-1}\mathbf{M }(\mathbf{I } +\alpha \mathbf{N }^\mathrm{T}\mathbf{B }^{-1}\mathbf{M })^{-1}\mathbf{N }^\mathrm{T}\mathbf{B }^{-1} \end{aligned} \end{aligned}$$

(33)

In the case that $\mathbf{B } = \mathbf{I }$, it obtains that $(\mathbf{I }+\frac{\tau }{2}\mathbf{A })^{-1}=\mathbf{I }-\frac{\tau }{2}\mathbf{M }(\mathbf{I }+\frac{\tau }{2}\mathbf{N }^\mathrm{T}\mathbf{M })^{-1}\mathbf{N }^\mathrm{T}$. As $\mathbf{I }-\frac{\tau }{2}\mathbf{A }=\mathbf{I }-\frac{\tau }{2}\mathbf{MN }^\mathrm{T}$, we have

$$\begin{aligned} y(\tau )= & {} x-\frac{\tau }{2}\mathbf{M }\left( \left( \mathbf{I }+ \frac{\tau }{2}\mathbf{N }^\mathrm{T}\mathbf{M }\right) ^{-1} \left( \mathbf{I }-\frac{\tau }{2}\mathbf{N }^\mathrm{T}\mathbf{M }\right) +\mathbf{I }\right) \mathbf{N }^\mathrm{T}x \\= & {} x-\tau \mathbf{M } \left( \mathbf{I }+\frac{\tau }{2}\mathbf{N }^\mathrm{T} \mathbf{M }\right) ^{-1}\mathbf{N }^\mathrm{T}x \end{aligned}$$

Part (2): When $\mathbf{A }=\mathbf{a }\mathbf{x }^\mathrm{T}-\mathbf{x }\mathbf{a }^\mathrm{T}$, then $\mathbf{M }=[\mathbf{a },\mathbf{x }]$ and $\mathbf{N }=[{\mathbf{a }},-\mathbf{a }]$, let $\mathbf{S }=\mathbf{I }+\frac{\tau }{2}\mathbf{N }^\mathrm{T}\mathbf{M }$,

$$\begin{aligned} \mathbf{S }= & {} \left[ \begin{array}{cc} 1 &{}\quad 0 \\ 0 &{}\quad 1 \end{array} \right] -\frac{\tau }{2} \left[ \begin{array}{cc} \mathbf{x }^\mathrm{T}\mathbf{a } &{}\quad \mathbf{x }^\mathrm{T}\mathbf{x } \\ -\mathbf{a }^\mathrm{T}\mathbf{a } &{}\quad -\mathbf{a }^\mathrm{T}\mathbf{x } \end{array} \right] \nonumber \\= & {} \left[ \begin{array}{cc} 1-\frac{\tau }{2}\mathbf{x }^\mathrm{T}\mathbf{a }&{}\quad -\frac{\tau }{2}\mathbf{x }^\mathrm{T}\mathbf{x }\\ [.5pc] \frac{\tau }{2}\mathbf{a }^\mathrm{T}\mathbf{a }&{}\quad 1+\frac{\tau }{2}\mathbf{a }^\mathrm{T}\mathbf{x } \end{array} \right] \end{aligned}$$

(34)

$$\begin{aligned} y(\tau )= & {} \mathbf{x }-\tau \left[ \begin{array}{ll} \mathbf{a }&{}\quad \mathbf{x }\\ \end{array} \right] \mathbf{S }^{-1}\left[ \begin{array}{c} \mathbf{x }^\mathrm{T}\\ -\mathbf{a }^\mathrm{T} \end{array} \right] \mathbf{x } \end{aligned}$$

(35)

Since $\mathbf{S }^{-1}=\frac{\mathbf{S }^{*}}{|\mathbf{S }|}$, and

$$\begin{aligned} \mathbf{S }^{*}= & {} \left[ \begin{array}{cc} 1+\frac{\tau }{2}\mathbf{a }^\mathrm{T}\mathbf{x } &{}\quad \frac{\tau }{2}\mathbf{x }^\mathrm{T}\mathbf{x }\\ -\frac{\tau }{2}\mathbf{a }^\mathrm{T}\mathbf{a } &{}\quad 1-\frac{\tau }{2}\mathbf{x }^\mathrm{T}\mathbf{a } \end{array} \right] \end{aligned}$$

(36)

$$\begin{aligned} |\mathbf{S }|= & {} \left( 1+\frac{\tau }{2}\mathbf{a }^\mathrm{T}\mathbf{x }\right) \left( 1-\frac{\tau }{2}\mathbf{x }^\mathrm{T}\mathbf{a }\right) - \left( -\frac{\tau }{2}\mathbf{x }^\mathrm{T}\mathbf{x }\right) \left( \frac{\tau }{2}\mathbf{a }^\mathrm{T}\mathbf{a }\right) \nonumber \\= & {} 1-\left( \frac{\tau }{2}\right) ^{2}(\mathbf{a }^\mathrm{T}\mathbf{x })^{2}+ \left( \frac{\tau }{2}\right) ^{2}\parallel \mathbf{a }\parallel _{2}^{2}\parallel \mathbf{x }\parallel _{2}^{2} \end{aligned}$$

(37)

Let $k=1-(\frac{\tau }{2})^{2}(\mathbf{a }^\mathrm{T}\mathbf{x })^{2}+ (\frac{\tau }{2})^{2}\parallel \mathbf{a }\parallel _{2}^{2}\parallel \mathbf{x }\parallel _{2}^{2}$, then

$$\begin{aligned} y(\tau )=\mathbf{x }-\tau \left[ \begin{array}{ll} \mathbf{a }&{}\quad \mathbf{x }\\ \end{array} \right] \frac{\mathbf{S }^{*}}{|\mathbf{S }|}\left[ \begin{array}{c} \mathbf{x }^\mathrm{T}\\ -\mathbf{a }^\mathrm{T} \end{array} \right] \mathbf{x } \end{aligned}$$

(38)

Using (36–38), one can obtain (15). $\square$