An automatic feature generation approach to multiple instance learning and its applications to image databases

Abstract

Automatic content-based image categorization is a challenging research topic and has many practical applications. Images are usually represented as bags of feature vectors, and the categorization problem is studied in the Multiple-Instance Learning (MIL) framework. In this paper, we propose a novel learning technique which transforms the MIL problem into a standard supervised learning problem by defining a feature vector for each image bag. Specifically, the feature vectors of the image bags are grouped into clusters and each cluster is given a label. Using these labels, each instance of an image bag can be replaced by a corresponding label to obtain a bag of cluster labels. Data mining can then be employed to uncover common label patterns for each image category. These label patterns are converted into bags of feature vectors; and they are used to transform each image bag in the data set into a feature vector such that each vector element is the distance of the image bag to a distinct pattern bag. With this new image representation, standard supervised learning algorithms can be applied to classify the images into the pre-defined categories. Our experimental results demonstrate the superiority of the proposed technique in categorization accuracy as compared to state-of-the-art methods.

Appendix: Proof of convergence of Algorithm 1

In this section, we would like to prove the correctness of Algorithm 1. The problem to be solved is to find an optimal vector $\vec{v}^{j, c_i}$ satisfying,

$$ \vec{v}^{j, c_i} = {\arg\min}_{\vec{v} \in \Re^d} \sum\limits_{Y^{j, c_i}_k \in \mathcal{Y}^{j, c_i}} \min\limits_{\vec{x}_{kl} \in Y^{j, c_i}_k} d_2(\vec{v}, \vec{x}_{kl}), $$

in which, $\mathcal{Y}^{j, c_i}$ is a set of bags with regard to cluster c _i and category j.

For a vector $\vec{v}$, and a mapping function $\mathit \Upsilon$, denote $\mathit \Upsilon(Y^{j, c_i}_k)$ to be the instance from the bag $Y^{j, c_i}_k$ which is mapped to vector $\vec{v}$. Therefore the optimal $\vec{v}^{j, c_i}$ also achieves the below optimization problem,

$$ \vec{v}^{j, c_i} = {\arg\min}_{\vec{v}, \mathit \Upsilon} \sum\limits_{Y^{j, c_i}_k \in \mathcal{Y}^{j, c_i}} d_2(\vec{v}, \mathit \Upsilon(Y^{j, c_i}_k)). $$

Therefore, we would like to find a vector to minimize

$$ \label{eqn:objfunc} f(\vec{v}, \mathit \Upsilon) = \sum\limits_{Y^{j, c_i}_k \in \mathcal{Y}^{j, c_i}} d_2(\vec{v}, \mathit \Upsilon(Y^{j, c_i}_k)). $$

For a fixed vector $\vec{v}$, the optimal mapping of each bag $Y^{j, c_i}_k$ is to map $\vec{v}$ to the closest one among all the instances in $Y^{j, c_i}_k$, i.e.,

$$ \mathit \Upsilon(Y^{j, c_i}_k) = {\arg\min}_{\vec{x} \in Y^{j, c_i}_k} d_2(\vec{x}, \vec{v}). $$

For a fixed mapping $\mathit \Upsilon$, the optimal vector is the centers of all the matched instances, i.e.,

$$ \vec{v} = \frac{1}{|\mathcal{Y}^{j, c_i}|} \sum\limits_{Y^{j, c_i}_k \in \mathcal{Y}^{j, c_i}} \mathit \Upsilon(Y^{j, c_i}_k). $$

Algorithm 1 takes the iterative approach to reach a local minimum of the objective function defined in Eq. 1. It starts with an initial guess of the vector. Each run of Step 2 of the algorithm is to find the matched instances with regard to the current vector. This guarantees to reduce the objective. In Step 3, the vector to be computed is updated as the centroid of the matched instances, which certainly decreases the objective value. Therefore Algorithm 1 is sure to have the objective value smaller and smaller. Because there are only a finite number of instances in $\mathcal{Y}^{j, c_i}$, there only exists a finite number of mapping, and the objective function defined in Eq. 1 is lower-bounded. Overall, Algorithm 1 guarantees to converge and the derived optimal vector gives a local minimum of the objective function.