The Support Vector Machine

Abstract

Three separate sections comprise this chapter. The first presents an overview of statistical learning theory (SLT) as applied to machine learning. The topics covered are empirical or true risk minimization, the risk minimization principle (RMP), theoretical concept of risk minimization, function f₀(X) that minimizes the expected (or true) risk, asymptotic consistency or uniform convergence, an example of the generalized bound for binary classification and finally, how are learning machines formed.

The second part of the chapter covers the theory of support vector machine (SVM) learning theory and develops solutions for the following three classes of problems:

• Linear separable systems

• Linear non-separable systems

• And non-linear, non-separable systems

It then introduces the topic of kernels, what they are, and how they might be chosen. A brief pointer is provided to the SVM literature available on the web.

These sections are followed by a sketch of how the SVM may be hybridized with the GA for feature subset selection and points the way to the value of further hybridization with an ensemble approach, the topic of the next chapter.