Multi-label Learning with Missing Labels Using Mixed Dependency Graphs

Abstract

This work focuses on the problem of multi-label learning with missing labels (MLML), which aims to label each test instance with multiple class labels given training instances that have an incomplete/partial set of these labels (i.e., some of their labels are missing). The key point to handle missing labels is propagating the label information from the provided labels to missing labels, through a dependency graph that each label of each instance is treated as a node. We build this graph by utilizing different types of label dependencies. Specifically, the instance-level similarity is served as undirected edges to connect the label nodes across different instances and the semantic label hierarchy is used as directed edges to connect different classes. This base graph is referred to as the mixed dependency graph, as it includes both undirected and directed edges. Furthermore, we present another two types of label dependencies to connect the label nodes across different classes. One is the class co-occurrence, which is also encoded as undirected edges. Combining with the above base graph, we obtain a new mixed graph, called mixed graph with co-occurrence (MG-CO). The other is the sparse and low rank decomposition of the whole label matrix, to embed high-order dependencies over all labels. Combining with the base graph, the new mixed graph is called as MG-SL (mixed graph with sparse and low rank decomposition). Based on MG-CO and MG-SL, we further propose two convex transductive formulations of the MLML problem, denoted as MLMG-CO and MLMG-SL respectively. In both formulations, the instance-level similarity is embedded through a quadratic smoothness term, while the semantic label hierarchy is used as a linear constraint. In MLMG-CO, the class co-occurrence is also formulated as a quadratic smoothness term, while the sparse and low rank decomposition is incorporated into MLMG-SL, through two additional matrices (one is assumed as sparse, and the other is assumed as low rank) and an equivalence constraint between the summation of this two matrices and the original label matrix. Interestingly, two important applications, including image annotation and tag based image retrieval, can be jointly handled using our proposed methods. Experimental results on several benchmark datasets show that our methods lead to significant improvements in performance and robustness to missing labels over the state-of-the-art methods.

Convexity Proof

For clarity and continuity, we rewrite the continuous optimization problem of MLMG-CO here,

$$\begin{aligned} \min _\mathbf {Z}&f_1(\mathbf {Z}) = -\mathrm {tr}( \overline{\mathbf {Y}}^\top \mathbf {Z}) + \beta \mathrm {tr}(\mathbf {Z}\mathbf {L}_\mathbf {X}\mathbf {Z}^\top ) + \gamma \mathrm {tr}(\mathbf {Z}^\top \mathbf {L}_{\mathbf {C}} \mathbf {Z}),\nonumber \\ \text {s.t.}&\mathbf {Z}\in [0, 1]^{m \times n}, \quad \varvec{\varPhi }^\top \mathbf {Z}\ge 0. \end{aligned}$$

(28)

Firstly we introduce the following vector variables:

$$\begin{aligned}&\mathbf {z}= \text {vec}(\mathbf {Z}) = [\mathbf {Z}_{11}, \ldots , \mathbf {Z}_{m1}, \ldots , \mathbf {Z}_{mn}]^\top \in \{ -1, +1 \}^{mn \times 1}, \nonumber \\&\overline{\mathbf {y}} = \text {vec}(\overline{\mathbf {Y}})= [\overline{\mathbf {Y}}_{11}, \ldots , \overline{\mathbf {Y}}_{m1}, \ldots , \overline{\mathbf {Y}}_{mn}]^\top \in \{ -1, 0, +1 \}^{mn}, \nonumber \\&\mathbf {W}= \beta \cdot \mathbf {W}_\mathbf {X}^\top \otimes \mathbf {I}_m + \gamma \cdot \mathbf {I}_n \otimes \mathbf {W}_{\mathbf {C}}\in \mathbb {R}^{mn \times mn}, \nonumber \\&\mathbf {L} = \beta \cdot \mathbf {L}_\mathbf {X}^\top \otimes \mathbf {I}_m + \gamma \cdot \mathbf {I}_n \otimes \mathbf {L}_{\mathbf {C}} \in \mathbb {R}^{mn \times mn}, \nonumber \\&\overline{\varvec{\varPhi }} = \varvec{\varPhi } \otimes \mathbf {I}_n, \end{aligned}$$

where $\otimes$ indicates the Kronecker product (Zehfuss 1858). Then Problem (28) can be transformed to its equivalent vector based formulation, as follows:

$$\begin{aligned} \arg \min _\mathbf {z}&\quad f_2(\mathbf {z}) = -\overline{\mathbf {y}}^\top \mathbf {z}+ \mathbf {z}^\top \mathbf {L} \mathbf {z}, \nonumber \\ \text {s.t.}&\quad \mathbf {z}\in [0, 1]^{mn \times 1}, \quad \overline{\varvec{\varPhi }}^\top \mathbf {z}\ge 0. \end{aligned}$$

(29)

Lemma 1

$\mathbf {L}$ is positive semi-definite (PSD).

Proof

Given two square matrix $\mathbf {A} \in \mathbb {R}^{n_1 \times n_1}$ and $\mathbf {B} \in \mathbb {R}^{n_2 \times n_2}$, their eigenvalues are denoted as $\lambda _1,\ldots ,\lambda _{n_1}$ and $\mu _1,\ldots ,\mu _{n_2}$. According to the property of Kronecker product, the eigenvalues of $\mathbf {A} \otimes \mathbf B$ are $\lambda _i \mu _j, i=1,\ldots ,n_1; j = 1,\ldots ,n_2$. $\mathbf {L}_\mathbf {X}^\top$ is PSD and $\mathbf {I}_m$ is positive definite (PD). Obviously all eigenvalues of $\mathbf {L}_\mathbf {X}^\top \otimes \mathbf {I}_m$ are non-zero values, so $\mathbf {L}_\mathbf {X}^\top \otimes \mathbf {I}_m$ is a PSD matrix. Similarly we can obtain that $\mathbf {I}_n \otimes \mathbf {L}_{\mathbf {C}}$ is also PSD. Finally, as $\mathbf {L}$ is the positive weighted linear combination of two PSD matrices, it is easy to conclude that $\mathbf {L}$ is a PSD matrix. $\square$

Proposition 1

Problem (28) is convex.

Proof

The Hessian of the objective function in (29) with respect to $\mathbf {z}$ is $\mathbf {L}$, which has been proven to be a PSD matrix in Lemma 1. Thus, the objective function (29) is a convex function in $\mathbf {z}$. The box and linear inequality constraints lead to a convex feasible solution space (that satisfies Slater’s condition), so it is easy to conclude that Problem (29) is a convex optimization problem. Finally, as Problems (28) and (29) are equivalent, thus Problem (28) is a convex problem. $\square$