A Comparative Study of $\text {L}_1$ and $\text {L}_2$ Norms in Support Vector Data Descriptions

Abstract

The Support Vector Data Description ($\text {L}_1$ SVDD) is a non-parametric one-class classification algorithm that utilizes the $\text {L}_1$ norm in its objective function. An alternative formulation of SVDD, called $\text {L}_2$ SVDD, uses a $\text {L}_2$ norm in its objective function and has not been extensively studied. $\text {L}_1$ SVDD and $\text {L}_2$ SVDD are formulated as distinct quadratic programming (QP) problems and can be solved with a QP-solver. The $\text {L}_2$ SVDD and $\text {L}_1$ SVDD’s ability to detect small and large shifts in data generated from multivariate normal, multivariate t, and multivariate Laplace distributions is evaluated. Similar comparisons are made using real-world datasets taken from various applications including oncology, activity recognition, marine biology, and agriculture. In both the simulated and real-world examples, $\text {L}_2$ SVDD and $\text {L}_1$ SVDD perform similarly, though, in some cases, one outperforms the other. We propose an extension of the SMO algorithm for $\text {L}_2$ SVDD, and we compare the runtimes of four algorithms: $\text {L}_2$ SVDD (SMO), $\text {L}_2$ SVDD (QP), $\text {L}_1$ SVDD (SMO), and $\text {L}_1$ SVDD (QP). The runtimes favor $\text {L}_1$ SVDD (QP) versus $\text {L}_2$ SVDD (QP), sometimes substantially; however using SMO reduces the difference in runtimes considerably, making $\text {L}_2$ SVDD (SMO) feasible for practical applications. We also present gradient descent and stochastic gradient descent algorithms for linear versions of both the $\text {L}_1$ SVDD and $\text {L}_2$ SVDD. Examples using simulated and real-world data show that both methods perform similarly. Finally, we apply the $\text {L}_1$ SVDD and $\text {L}_2$ SVDD to a real-world dataset that involves monitoring machine failures in a manufacturing process.