Advertisement

Linear Projection Learned from Hybrid CKA for Enhancing Distance-Based Classifiers

  • Diego Collazos-HuertasEmail author
  • David Cárdenas-Peña
  • German Castellanos-Dominguez
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11401)

Abstract

Most machine learning approaches are classified into either supervised or unsupervised. However, joining generative and discriminative functions in the learning process may beneficially influence each other. Using the centered kernel alignment similarity, this paper proposes a new hybrid cost function based on the linear combination of two computed terms: a discriminative component that accounts for the affinity between projected data and their labels, and a generative component that measures the similarity between the input and projected distributions. Further, the data projection is assumed as a linear model so that a matrix has to be learned by maximizing the proposed cost function. We compare our approach using a kNN classifier against the raw features and a multi-layer perceptron machine. Attained results on a handwritten digit recognition database show that there exists a trade-off value other than the trivial ones that provide the highest accuracy. Moreover, the proposed approach not only outperforms the baseline machines but also becomes more robust to several noise levels.

Keywords

Centered kernel aligment Projection learning Hybrid cost function 

Notes

Acknowledgment

This work was supported by Doctorados Nacionales 2017 - Conv 785 and the research project 111974454838, both funded by COLCIENCIAS.

References

  1. 1.
    Alvarez-Meza, A., Lee, J., Verleysen, M., Castellanos-Dominguez, G.: Kernel-based dimensionality reduction using renyi’s \(\alpha \)-entropy measures of similarity. Neurocomputing 222, 36–46 (2017).  https://doi.org/10.1016/j.neucom.2016.10.004Google Scholar
  2. 2.
    Álvarez-Meza, A.M., Cárdenas-Peña, D., Castellanos-Dominguez, G.: Unsupervised kernel function building using maximization of information potential variability. In: Bayro-Corrochano, E., Hancock, E. (eds.) CIARP 2014. LNCS, vol. 8827, pp. 335–342. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-12568-8_41CrossRefGoogle Scholar
  3. 3.
    Brockmeier, A.J., Choi, J.S., Kriminger, E.G., Francis, J.T., Principe, J.C.: Neural decoding with kernel-based metric learning. Neural Comput. 26(6), 1080–1107 (2014).  https://doi.org/10.1162/NECO_a_00591MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Brockmeier, A.J., et al.: Information-theoretic metric learning: 2-D linear projections of neural data for visualization. In: 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 5586–5589. IEEE (2013).  https://doi.org/10.1109/EMBC.2013.6610816
  5. 5.
    Cárdenas-Peña, D., Collazos-Huertas, D., Álvarez-Meza, A., Castellanos-Dominguez, G.: Supervised kernel approach for automated learning using general stochastic networks. Eng. Appl. Artif. Intell. 68, 10–17 (2018).  https://doi.org/10.1016/j.engappai.2017.10.003CrossRefGoogle Scholar
  6. 6.
    Cárdenas-Peña, D., Collazos-Huertas, D., Castellanos-Dominguez, G.: Centered kernel alignment enhancing neural network pretraining for mri-based dementia diagnosis. Comput. Math. Methods Med. 2016, 1–10 (2016).  https://doi.org/10.1155/2016/9523849MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Cortes, C., Mohri, M., Rostamizadeh, A.: Algorithms for learning kernels based on centered alignment. J. Mach. Learn. Res. 13, 795–828 (2012). http://dl.acm.org/citation.cfm?id=2503308.2188413MathSciNetzbMATHGoogle Scholar
  8. 8.
    Cowley, B.R., et al.: DataHigh: graphical user interface for visualizing and interacting with high-dimensional neural activity. J. Neural Eng. 10(6), 066012 (2013).  https://doi.org/10.1088/1741-2560/10/6/066012CrossRefGoogle Scholar
  9. 9.
    Harandi, M., Salzmann, M., Hartley, R.: Dimensionality reduction on spd manifolds: the emergence of geometry-aware methods. IEEE Trans. Pattern Anal. Mach. Intell. 40(1), 48–62 (2018).  https://doi.org/10.1109/TPAMI.2017.2655048CrossRefGoogle Scholar
  10. 10.
    Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.R.: Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580 (2012)
  11. 11.
    Zöhrer, M., Pernkopf, F.: General stochastic networks for classification. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27, pp. 2015–2023. Curran Associates, Inc. (2014). http://dl.acm.org/citation.cfm?id=2969033.2969052

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Diego Collazos-Huertas
    • 1
    Email author
  • David Cárdenas-Peña
    • 1
  • German Castellanos-Dominguez
    • 1
  1. 1.Signal Processing and Recognition GroupUniversidad Nacional de ColombiaManizalesColombia

Personalised recommendations