Advertisement

Improving SNR and Reducing Training Time of Classifiers in Large Datasets via Kernel Averaging

  • Matthias S. Treder
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11309)

Abstract

Kernel methods are of growing importance in neuroscience research. As an elegant extension of linear methods, they are able to model complex non-linear relationships. However, since the kernel matrix grows with data size, the training of classifiers is computationally demanding in large datasets. Here, a technique developed for linear classifiers is extended to kernel methods: In linearly separable data, replacing sets of instances by their averages improves signal-to-noise ratio (SNR) and reduces data size. In kernel methods, data is linearly non-separable in input space, but linearly separable in the high-dimensional feature space that kernel methods implicitly operate in. It is shown that a classifier can be efficiently trained on instances averaged in feature space by averaging entries in the kernel matrix. Using artificial and publicly available data, it is shown that kernel averaging improves classification performance substantially and reduces training time, even in non-linearly separable data.

Keywords

Kernel Machine learning Big data SVM FDA 

References

  1. 1.
    Ayres-de Campos, D., Bernardes, J., Garrido, A., Marques-de Sá, J., Pereira-Leite, L.: Sisporto 2.0: a program for automated analysis of cardiotocograms. J. Matern. Fetal Med. 9(5), 311–318 (2000).  https://doi.org/10.1002/1520-6661(200009/10)9:5<311::AID-MFM12>3.0.CO;2-9Google Scholar
  2. 2.
    Chang, C.C, Lin, C.J.: LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 1–27 (2011)CrossRefGoogle Scholar
  3. 3.
    Choudhury, S., Fishman, J.R., McGowan, M.L., Juengst, E.T.: Big data, open science and the brain: lessons learned from genomics. Front. Hum. Neurosci. 8, 239 (2014).  https://doi.org/10.3389/fnhum.2014.00239CrossRefGoogle Scholar
  4. 4.
    Cichy, R.M., Pantazis, D.: Multivariate pattern analysis of MEG and EEG: a comparison of representational structure in time and space. NeuroImage 158, 441–454 (2017).  https://doi.org/10.1016/j.neuroimage.2017.07.023CrossRefGoogle Scholar
  5. 5.
    Cichy, R.M., Ramirez, F.M., Pantazis, D.: Can visual information encoded in cortical columns be decoded from magnetoencephalography data in humans? NeuroImage 121, 193–204 (2015).  https://doi.org/10.1016/j.neuroimage.2015.07.011CrossRefGoogle Scholar
  6. 6.
    Danziger, S.A., et al.: Predicting positive p53 cancer rescue regions using most informative positive (MIP) active learning. PLoS Comput. Biol. 5(9), e1000498 (2009).  https://doi.org/10.1371/journal.pcbi.1000498MathSciNetCrossRefGoogle Scholar
  7. 7.
    Dima, D.C., Perry, G., Singh, K.D.: Spatial frequency supports the emergence of categorical representations in visual cortex during natural scene perception. NeuroImage 179, 102–116 (2018).  https://doi.org/10.1016/J.NEUROIMAGE.2018.06.033CrossRefGoogle Scholar
  8. 8.
    Gonzalez-Moreno, A., et al.: Signal-to-noise ratio of the MEG signal after preprocessing. J. Neurosci. Methods 222, 56–61 (2014).  https://doi.org/10.1016/J.JNEUMETH.2013.10.019CrossRefGoogle Scholar
  9. 9.
    Hainmueller, J., Hazlett, C., Alvarez, R.M.: Kernel regularized least squares: reducing misspecification bias with a flexible and interpretable machine learning approach. Polit. Anal. 22(2), 143–168 (2014).  https://doi.org/10.1093/pan/mpt019CrossRefGoogle Scholar
  10. 10.
    Hinton, G.E.: Machine learning for neuroscience. Neural Syst. Circ. 1(1), 12 (2011).  https://doi.org/10.1186/2042-1001-1-12CrossRefGoogle Scholar
  11. 11.
    Hwang, H.J., et al.: A gaze independent brain-computer interface based on visual stimulation through closed eyelids. Sci. Rep. 5, 15890 (2015).  https://doi.org/10.1038/srep15890CrossRefGoogle Scholar
  12. 12.
    Jäkel, F., Schölkopf, B., Wichmann, F.A.: Does cognitive science need kernels? Trends Cogn. Sci. 13, 381–388 (2009). https://www.sciencedirect.com/science/article/pii/S1364661309001430CrossRefGoogle Scholar
  13. 13.
    Orrù, G., Pettersson-Yeo, W., Marquand, A.F., Sartori, G., Mechelli, A.: Using support vector machine to identify imaging biomarkers of neurological and psychiatric disease: a critical review. Neurosci. Biobehav. Rev. 36(4), 1140–1152 (2012).  https://doi.org/10.1016/j.neubiorev.2012.01.004CrossRefGoogle Scholar
  14. 14.
    Schölkopf, B., Smola, A.J.: A short introduction to learning with kernels. In: Mendelson, S., Smola, A.J. (eds.) Advanced Lectures on Machine Learning. Lecture Notes in Computer Science, vol. 2600, pp. 41–64. Springer, Heidelberg (2003).  https://doi.org/10.1007/3-540-36434-X_2CrossRefzbMATHGoogle Scholar
  15. 15.
    Schrouff, J., et al.: PRoNTo: pattern recognition for neuroimaging toolbox. Neuroinformatics 11(3), 319–337 (2013).  https://doi.org/10.1007/s12021-013-9178-1CrossRefGoogle Scholar
  16. 16.
    Schrouff, J., Mourão-Miranda, J., Phillips, C., Parvizi, J.: Decoding intracranial EEG data with multiple kernel learning method. J. Neurosci. Methods 261, 19–28 (2016).  https://doi.org/10.1016/J.JNEUMETH.2015.11.028CrossRefGoogle Scholar
  17. 17.
    Treder, M.S., Purwins, H., Miklody, D., Sturm, I., Blankertz, B.: Decoding auditory attention to instruments in polyphonic music using single-trial EEG classification. J. Neural Eng. 11(2), 026009 (2014).  https://doi.org/10.1088/1741-2560/11/2/026009CrossRefGoogle Scholar
  18. 18.
    Wang, X., Xing, E.P., Schaid, D.J.: Kernel methods for large-scale genomic data analysis. Brief. Bioinf. 16(2), 183–192 (2015).  https://doi.org/10.1093/bib/bbu024CrossRefGoogle Scholar
  19. 19.
    Weinstein, J.N., et al.: The cancer genome atlas pan-cancer analysis project. Nat. Genet. 45(10), 1113–1120 (2013).  https://doi.org/10.1038/ng.2764CrossRefGoogle Scholar
  20. 20.
    Youssofzadeh, V., McGuinness, B., Maguire, L.P., Wong-Lin, K.: Multi-kernel learning with dartel improves combined MRI-PET classification of Alzheimer’s disease in AIBL data: group and individual analyses. Front. Hum. Neurosci. 11, 380 (2017).  https://doi.org/10.3389/fnhum.2017.00380CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.School of Computer Science and InformaticsCardiff UniversityCardiffUK

Personalised recommendations