Boosted Classifiers for Antitank Mine Detection in C-Scans from Ground-Penetrating Radar

Abstract

We investigate the problem of automatic antitank mine detection. Subject to detection are 3D images, so-called C-scans, generated by a GPR (Ground-Penetrating Radar) system of our construction. In the paper we focus on boosting as a machine learning approach well suited for large-scale data such as GPR data. We compare five variants of weak learners with real-valued responses trained by the same boosting scheme. Three of the variants are single-feature-based learners that differ in the way they approximate class conditional distributions. The two remaining variants are shallow decision trees, respectively, with four and eight terminal nodes, introducing joint-feature conditionals.

This project has been partially financed by the Polish Ministry of Science and Higher Education; agreement no. 0091/R/TOO/2010/12 for R&D project no. 0 R00 0091 12, dated on 30.11.2010, carried out by a consortium of Military Institute of Armament Technology in Zielonka, and Autocomp Management in Szczecin, Poland.

Appendix A: Notes on Boosting’s Connection to Logistic Regression

Following [3], we point out some important properties of boosting in the context of its connection to logistic regression and additive modeling.

Think of the unknown joint probability distribution (population) from which the data is drawn. Let $p(\mathbf {x},y)$ denote its density function, which can also be expressed as $p(\mathbf {x},y)=P(y|\mathbf {x})p(\mathbf {x})$. Now, consider the exponential criterion for some classifier function $F$ defined as an expectation taken with respect to $p$:

$$\begin{aligned} Q_p(F)=\mathbb {E}_p\bigl (e^{-yF(\mathbf {x})}\bigr )&= \int _\mathbf {x}\sum _{y\in \{-1,1\}}e^{-yF(\mathbf {x})}p(\mathbf {x},y)\,\mathbf {dx}\nonumber \\&=\int _\mathbf {x}\Bigl (P(y{=}{-1}|\mathbf {x})e^{F(\mathbf {x})}{+}P(y{=}1|\mathbf {x})e^{-F(\mathbf {x})}\Bigr )p(\mathbf {x})\,\mathbf {dx}. \end{aligned}$$

(12)

To minimize $Q_p$ we demand that $\partial Q_p(F)/\partial F=0$ (in fact it suffices to minimize the inner conditional expectation) and obtain the formula for the minimizer $F^*(\mathbf {x}) = 1/2 \log \left( P(y{=}1|\mathbf {x})/P(y{=}{-1}|\mathbf {x})\right)$ being half the logit transform, typical for logistic regression. If an algorithm could somehow immediately find the perfect function $F^*$, then the boosting procedure could be stopped after just one round. In practice, weak learners are rough approximations of $F^*$ and multiple rounds are needed. Solving $F^*$ for probability leads to a form of sigmoid:

$$\begin{aligned} P(y=1|\mathbf {x})=e^{2F^*(\mathbf {x})}\Big /\left( 1+e^{2F^*(\mathbf {x})}\right) =1\Big /\left( 1+e^{-2F^*(\mathbf {x})}\right) , \end{aligned}$$

(13)

and again the similarity to logistic regression can be seen. The two minor differences are: the factor of $2$ in the exponent, and the fact that in the traditional logistic regression one approximates $F^*$ by a linear model $F^*(\mathbf {x})\approx a_0+a_1 x_1 + \cdots + a_n x_n$, whereas in boosting it is a linear combination of weak learners, i.e., $F^*(\mathbf {x})\approx \sum _k f_k(\mathbf {x})$, so in general arbitrary but simple functions, each being possibly a function of all variables.

Think of the error residuals technique, known from regression modeling in general. In an additive model built sequentially, each successive piece of approximation “explains" some part of the target quantity and that part is subtracted from the target, so that future pieces can focus on residuals. Boosting’s reweighing scheme proceeds in an akin manner. Suppose we have fixed a partial model $F$ and would like to update it, i.e., to have $F:=F+f$. Recall the data-based reweighing formulas (2). Let us define their population-based counterparts:

$$\begin{aligned} Z=\int _\mathbf {x}\sum _{y\in \{-1,1\}}e^{-yF(\mathbf {x})}p(\mathbf {x},y)\,\mathbf {dx},\qquad w(\mathbf {x},y)=e^{-y F(\mathbf {x)}} p(\mathbf {x},y) / Z. \end{aligned}$$

(14)

Note that $Z$ works as a normalizer but simultaneously is our exponential criterion value for the model so far—$Q_p(F)$. Now, consider the criterion at $F+f$:

$$\begin{aligned} Q_p(F{+}f)&={\int _\mathbf {x}\sum _{y\in \{-1,1\}}e^{-y\left( F(\mathbf {x})+f(\mathbf {x})\right) }p(\mathbf {x},y)\,\mathbf {dx}}\nonumber \\&={\int _\mathbf {x}\sum _{y\in \{-1,1\}}e^{-y f(\mathbf {x})}\underbrace{e^{-y F(\mathbf {x})}p(\mathbf {x},y)/Z}_{w(\mathbf {x},y)}\,\mathbf {dx}\cdot Z}{=}{Q_w(f) \cdot Q_p(F)}. \end{aligned}$$

(15)

Therefore, to minimize $Q(F+f)$ it suffices to greedily minimize $Q_w(f)$. The distribution $w$ indicates which “parts” of the target quantity are already explained.