Linear Projection Learned from Hybrid CKA for Enhancing Distance-Based Classifiers
Most machine learning approaches are classified into either supervised or unsupervised. However, joining generative and discriminative functions in the learning process may beneficially influence each other. Using the centered kernel alignment similarity, this paper proposes a new hybrid cost function based on the linear combination of two computed terms: a discriminative component that accounts for the affinity between projected data and their labels, and a generative component that measures the similarity between the input and projected distributions. Further, the data projection is assumed as a linear model so that a matrix has to be learned by maximizing the proposed cost function. We compare our approach using a kNN classifier against the raw features and a multi-layer perceptron machine. Attained results on a handwritten digit recognition database show that there exists a trade-off value other than the trivial ones that provide the highest accuracy. Moreover, the proposed approach not only outperforms the baseline machines but also becomes more robust to several noise levels.
KeywordsCentered kernel aligment Projection learning Hybrid cost function
This work was supported by Doctorados Nacionales 2017 - Conv 785 and the research project 111974454838, both funded by COLCIENCIAS.
- 2.Álvarez-Meza, A.M., Cárdenas-Peña, D., Castellanos-Dominguez, G.: Unsupervised kernel function building using maximization of information potential variability. In: Bayro-Corrochano, E., Hancock, E. (eds.) CIARP 2014. LNCS, vol. 8827, pp. 335–342. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-12568-8_41CrossRefGoogle Scholar
- 4.Brockmeier, A.J., et al.: Information-theoretic metric learning: 2-D linear projections of neural data for visualization. In: 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 5586–5589. IEEE (2013). https://doi.org/10.1109/EMBC.2013.6610816
- 10.Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.R.: Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580 (2012)
- 11.Zöhrer, M., Pernkopf, F.: General stochastic networks for classification. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27, pp. 2015–2023. Curran Associates, Inc. (2014). http://dl.acm.org/citation.cfm?id=2969033.2969052