SVM Classifier Fusion Using Belief Functions: Application to Hyperspectral Data Classification

Abstract

Hyperspectral imagery is a powerful source of information for recognition problems in a variety of fields. However, the resulting data volume is a challenge for classification methods especially considering industrial context requirements. Support Vector Machines (SVMs), commonly used classifiers for hyperspectral data, are originally suited for binary problems. Basing our study on [12] bbas allocation for binary classifiers, we investigate different strategies to combine two-class SVMs and tackle the multiclass problem. We evaluate the use of belief functions regarding the matter of SVM fusion with hyperspectral data for a waste sorting industrial application. We specifically highlight two possible ways of building a fast multi-class classifier using the belief functions framework that takes into account the process uncertainties and can use different information sources such as complementary spectra features.

Appendix: Evidential Calibration

Handling binary classifiers, the discernment frame is $\Omega ^{b}= \left\{ \left\{ 0\right\} ,\left\{ 1\right\} \right\}$. Then, for a given SVM having its own features in terms of number of samples, learning step performance, we aim at defining a belief function for each score that reflects the confidence we may have in each class. Indeed, this belief function will be used for forecasting taking into account the whole training set specificities. Explicitly, for each score s, the mass function, denoted m, is derived from the contour function: $\omega \rightarrow pl_X\left( \omega ,s\right)$, where $\omega =P\left( y=1\left| s\right. \right)$ (note that, $\omega$ is not a class but a probability).

To build the contour function on $\omega$, [12] uses the idea behind the logistic regression: approximating the probability $P\left( y=1\left| s\right. \right)$ by a sigmoid $sig_s\left( \theta \right) =\left[ 1+exp(\theta _0+\theta _1 s)\right] ^{-1}$, where the parameter $\hat{\theta }=\left( \theta _0, \theta _1\right)$ is determined by maximizing the likelihood function $L_X\left( \theta \right)$ over the training set $X=\{(s_1,y_1), \dots , (s_N,y_N) \}$ where, for each sample number i, $s_i\in \mathbb {R}$ is the score given by the considered classifier and $y_i\in \left\{ 0,1\right\}$ is its true label. Then, the contour function of interest is drawn for a given value of score. It derives from the 2D function plotting the relative value of the likelihood function $\frac{L_X(\theta )}{L_X(\hat{\theta })}$ versus $\theta =(\theta _0,\theta _1)$. Then, for any given pair $(s,\omega )$, the set of $\theta$ (i.e. $sig_s^{-1}(\omega )$) values is a straight line in $\mathbb {R}^2$. Then, the contour function value can be determined as the maximum value over this straight line:

$$\begin{aligned} pl^{\Omega _{j,k}^{b}}_X\left( \omega \right) =\left\{ \begin{array}{l@{\quad }l} 0 &{} \text {if }\omega \in \left\{ 0,1\right\} ,\\ \sup _{sig_s^{-1}(\omega )}\frac{L_X(\theta )}{L_X(\hat{\theta })} &{}\text {otherwise}, \end{array}\right. \end{aligned}$$

(7)

with $L_X\left( \theta \right) =\prod _{i=1}^N p_i^{y_i} (1-p_i)^{1-y_i}$ where $p_i=\frac{1}{1+exp\left( \theta _0+\theta _1 s_i\right) }$. Finally, from each $pl^{\Omega _{j,k}^{b}}_X$, the corresponding mass function $m^{\Omega _{j,k}^{b}}$ on binary discernment frame $\Omega _{j,k}^{b}$ is derived using the ‘likelihood based’ belief function for statistical inference approach proposed by Shafer and further justified by Denœux [4].