Class Prediction from Disparate Biological Data Sources Using an Iterative Multi-Kernel Algorithm

  • Yiming Ying
  • Colin Campbell
  • Theodoros Damoulas
  • Mark Girolami
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5780)


For many biomedical modelling tasks a number of different types of data may influence predictions made by the model. An established approach to pursuing supervised learning with multiple types of data is to encode these different types of data into separate kernels and use multiple kernel learning. In this paper we propose a simple iterative approach to multiple kernel learning (MKL), focusing on multi-class classification. This approach uses a block L 1-regularization term leading to a jointly convex formulation. It solves a standard multi-class classification problem for a single kernel, and then updates the kernel combinatorial coefficients based on mixed RKHS norms. As opposed to other MKL approaches, our iterative approach delivers a largely ignored message that MKL does not require sophisticated optimization methods while keeping competitive training times and accuracy across a variety of problems. We show that the proposed method outperforms state-of-the-art results on an important protein fold prediction dataset and gives competitive performance on a protein subcellular localization task.


Multiple kernel learning multi-class bioinformatics protein fold prediction protein subcellular localization 


  1. 1.
    Argyriou, A., Evgeniou, T., Pontil, M.: Multi-task feature learning. In: NIPS (2006)Google Scholar
  2. 2.
    Argyriou, A., Micchelli, C.A., Pontil, M., Ying, Y.: A spectral regularization framework for multi-task structure learning. In: NIPS (2007)Google Scholar
  3. 3.
    Bach, F., Lanckriet, G.R.G., Jordan, M.I.: Multiple kernel learning, conic duality and the SMO algorithm. In: ICML (2004)Google Scholar
  4. 4.
    Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)CrossRefGoogle Scholar
  5. 5.
    Damoulas, T., Girolami, M.: Probabilistic multi-class multi-kernel learning: On protein fold recognition and remote homology detection. Bioinformatics 24(10), 1264–1270 (2008)CrossRefPubMedGoogle Scholar
  6. 6.
    Ding, C., Dubchak, I.: Multi-class protein fold recognition using support vector machines and neural networks. Bioinformatics 17, 349–358 (2001)CrossRefPubMedGoogle Scholar
  7. 7.
    Gardy, J.L., et al.: PSORTb v.2.0: expanded prediction of bacterial protein subcellular localization and insights gained from comparative proteome analysis. Bioinformatics 21, 617–623 (2004)CrossRefPubMedGoogle Scholar
  8. 8.
    Girolami, M., Rogers, S.: Hierarchic Bayesian models for kernel learning. In: ICML (2005)Google Scholar
  9. 9.
    Lanckriet, G.R.G., Cristianini, N., Bartlett, P., Ghaoui, L.E., Jordan, M.I.: Learning the kernel matrix with semidefinite programming. J. of Machine Learning Research 5, 27–72 (2004)Google Scholar
  10. 10.
    Lin, Y., Zhang, H.: Component selection and smoothing in multivariate nonparametric regression. Annals of Statistics 34, 2272–2297 (2006)CrossRefGoogle Scholar
  11. 11.
    Micchelli, C.A., Pontil, M.: Learning the kernel function via regularization. J. of Machine Learning Research 6, 1099–1125 (2005)Google Scholar
  12. 12.
    Ong, C.S., Smola, A.J., Williamson, R.C.: Learning the kernel with hyperkernels. J. of Machine Learning Research 6, 1043–1071 (2005)Google Scholar
  13. 13.
    Rakotomamonjy, A., Bach, F., Canu, S., Grandvalet, Y.: More efficiency in multiple kernel learning. In: ICML (2007)Google Scholar
  14. 14.
    Rakotomamonjy, A., Bach, F., Canu, S., Grandvalet, Y.: SimpleMKL. J. of Machine Learning Research 9, 2491–2521 (2008)Google Scholar
  15. 15.
    Schölkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2002)Google Scholar
  16. 16.
    Shawe-Taylor, J., Cristianini, N.: Kernel methods for pattern analysis. Cambridge University Press, Cambridge (2004)CrossRefGoogle Scholar
  17. 17.
    Shen, H.B., Chou, K.C.: Ensemble classifier for protein fold pattern recognition. Bioinformatics 22, 1717–1722 (2006)CrossRefPubMedGoogle Scholar
  18. 18.
    Sonnenburg, S., Rätsch, G., Schäfer, C., Schölkopf, B.: Large scale multiple kernel learning. J. of Machine Learning Research 7, 1531–1565 (2006)Google Scholar
  19. 19.
  20. 20.
    Ye, J., Ji, S., Chen, J.: Multi-class discriminant kernel learning via convex programming. J. of Machine Learning Research 9, 719–758 (2008)Google Scholar
  21. 21.
    Zien, A., Ong, C.: Multi-class multiple kernel learning. In: ICML (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Yiming Ying
    • 1
  • Colin Campbell
    • 1
  • Theodoros Damoulas
    • 2
  • Mark Girolami
    • 2
  1. 1.Department of Engineering MathematicsUniversity of BristolBristolUnited Kingdom
  2. 2.Department of Computer ScienceUniversity of GlasgowGlasgowUnited Kingdom

Personalised recommendations