Linear Projection Learned from Hybrid CKA for Enhancing Distance-Based Classifiers

  • Diego Collazos-HuertasEmail author
  • David Cárdenas-Peña
  • German Castellanos-Dominguez
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11401)


Most machine learning approaches are classified into either supervised or unsupervised. However, joining generative and discriminative functions in the learning process may beneficially influence each other. Using the centered kernel alignment similarity, this paper proposes a new hybrid cost function based on the linear combination of two computed terms: a discriminative component that accounts for the affinity between projected data and their labels, and a generative component that measures the similarity between the input and projected distributions. Further, the data projection is assumed as a linear model so that a matrix has to be learned by maximizing the proposed cost function. We compare our approach using a kNN classifier against the raw features and a multi-layer perceptron machine. Attained results on a handwritten digit recognition database show that there exists a trade-off value other than the trivial ones that provide the highest accuracy. Moreover, the proposed approach not only outperforms the baseline machines but also becomes more robust to several noise levels.


Centered kernel aligment Projection learning Hybrid cost function 



This work was supported by Doctorados Nacionales 2017 - Conv 785 and the research project 111974454838, both funded by COLCIENCIAS.


  1. 1.
    Alvarez-Meza, A., Lee, J., Verleysen, M., Castellanos-Dominguez, G.: Kernel-based dimensionality reduction using renyi’s \(\alpha \)-entropy measures of similarity. Neurocomputing 222, 36–46 (2017). Scholar
  2. 2.
    Álvarez-Meza, A.M., Cárdenas-Peña, D., Castellanos-Dominguez, G.: Unsupervised kernel function building using maximization of information potential variability. In: Bayro-Corrochano, E., Hancock, E. (eds.) CIARP 2014. LNCS, vol. 8827, pp. 335–342. Springer, Cham (2014). Scholar
  3. 3.
    Brockmeier, A.J., Choi, J.S., Kriminger, E.G., Francis, J.T., Principe, J.C.: Neural decoding with kernel-based metric learning. Neural Comput. 26(6), 1080–1107 (2014). Scholar
  4. 4.
    Brockmeier, A.J., et al.: Information-theoretic metric learning: 2-D linear projections of neural data for visualization. In: 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 5586–5589. IEEE (2013).
  5. 5.
    Cárdenas-Peña, D., Collazos-Huertas, D., Álvarez-Meza, A., Castellanos-Dominguez, G.: Supervised kernel approach for automated learning using general stochastic networks. Eng. Appl. Artif. Intell. 68, 10–17 (2018). Scholar
  6. 6.
    Cárdenas-Peña, D., Collazos-Huertas, D., Castellanos-Dominguez, G.: Centered kernel alignment enhancing neural network pretraining for mri-based dementia diagnosis. Comput. Math. Methods Med. 2016, 1–10 (2016). Scholar
  7. 7.
    Cortes, C., Mohri, M., Rostamizadeh, A.: Algorithms for learning kernels based on centered alignment. J. Mach. Learn. Res. 13, 795–828 (2012). Scholar
  8. 8.
    Cowley, B.R., et al.: DataHigh: graphical user interface for visualizing and interacting with high-dimensional neural activity. J. Neural Eng. 10(6), 066012 (2013). Scholar
  9. 9.
    Harandi, M., Salzmann, M., Hartley, R.: Dimensionality reduction on spd manifolds: the emergence of geometry-aware methods. IEEE Trans. Pattern Anal. Mach. Intell. 40(1), 48–62 (2018). Scholar
  10. 10.
    Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.R.: Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580 (2012)
  11. 11.
    Zöhrer, M., Pernkopf, F.: General stochastic networks for classification. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27, pp. 2015–2023. Curran Associates, Inc. (2014).

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Diego Collazos-Huertas
    • 1
    Email author
  • David Cárdenas-Peña
    • 1
  • German Castellanos-Dominguez
    • 1
  1. 1.Signal Processing and Recognition GroupUniversidad Nacional de ColombiaManizalesColombia

Personalised recommendations