Kernel Dictionary Learning

Abstract

Sparse representations are linear by construction, a fact that can hinder their use in classification problems. Building vectors of characteristics from the signals to be classified can overcome the difficulties and is automated by employing kernels, which are functions that quantify the similarities between two vectors. DL can be extended to kernel form by assuming a specific form of the dictionary. DL algorithms have the usual form, comprising sparse coding and dictionary update. We present the kernel versions of OMP and of the most common update algorithms: MOD, SGK, AK-SVD, and K-SVD. The kernel methods use many operations involving a square kernel matrix whose size is equal to the number of signals; hence, their complexities are significantly higher than those of the standard methods. We present two ideas for reducing the size of the problem, the most prominent being that based on Nyström sampling. Finally, we show how kernel DL can be adapted to classification methods involving sparse representations, in particular SRC and discriminative DL.