Multi-task Drug Bioactivity Classification with Graph Labeling Ensembles

  • Hongyu Su
  • Juho Rousu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7036)


We present a new method for drug bioactivity classification based on learning an ensemble of multi-task classifiers. As the base classifiers of the ensemble we use Max-Margin Conditional Random Field (MMCRF) models, which have previously obtained the state-of-the-art accuracy in this problem. MMCRF relies on a graph structure coupling the set of tasks together, and thus turns the multi-task learning problem into a graph labeling problem. In our ensemble method the graphs of the base classifiers are random, constructed by random pairing or random spanning tree extraction over the set of tasks.

We compare the ensemble approaches on datasets containing the cancer inhibition potential of drug-like molecules against 60 cancer cell lines. In our experiments we find that ensembles based on random graphs surpass the accuracy of single SVM as well as a single MMCRF model relying on a graph built from auxiliary data.


drug bioactivity prediction multi-task learning ensemble methods kernel methods 


  1. 1.
    Breiman, L.: Bagging predictors. Machine Learning 24, 123–140 (1996)zbMATHGoogle Scholar
  2. 2.
    Ceroni, A., Costa, F., Frasconi, P.: Classification of small molecules by two- and three-dimensional decomposition kernels. Bioinformatics 23, 2038–2045 (2007)CrossRefGoogle Scholar
  3. 3.
    Dietterich, T.: Ensemble methods in machine learning. Multiple classifier systems, 1–15 (2000)Google Scholar
  4. 4.
    Esuli, A., Fagni, T., Sebastiani, F.: Boosting multi-label hierarchical text categorization. Information Retrieval 11(4), 287–313 (2008)CrossRefGoogle Scholar
  5. 5.
    Hastie, T., Tibshirani, R.: Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9(3), 432–441 (2008)CrossRefzbMATHGoogle Scholar
  6. 6.
    Meinshausen, N., Bühlmann, P., Zürich, E.: High dimensional graphs and variable selection with the lasso. Annals of Statistics 34, 1436–1462 (2006)CrossRefzbMATHGoogle Scholar
  7. 7.
    Obrezanova, O., Segall, M.D.: Gaussian processes for classification: Qsar modeling of admet and target activity. Journal of Chemical Information and Modeling 50(6), 1053–1061 (2010)CrossRefGoogle Scholar
  8. 8.
    Opitz, D., Maclin, R.: Popular ensemble methods: an empirical study. Journal of Artificial Intelligence Research 11, 169–198 (1999)zbMATHGoogle Scholar
  9. 9.
    Ralaivola, L., Swamidass, S., Saigo, H., Baldi, P.: Graph kernels for chemical informatics. Neural Networks 18, 1093–1110 (2005)CrossRefGoogle Scholar
  10. 10.
    Rousu, J., Saunders, C., Szedmak, S., Shawe-Taylor, J.: Kernel-Based Learning of Hierarchical Multilabel Classification Models. The Journal of Machine Learning Research 7, 1601–1626 (2006)zbMATHGoogle Scholar
  11. 11.
    Rousu, J., Saunders, C., Szedmak, S., Shawe-Taylor, J.: Efficient algorithms for max-margin structured classification. Predicting Structured Data, 105–129 (2007)Google Scholar
  12. 12.
    Schapire, R.E., Singer, Y.: Boostexter: A boosting-based system for text categorization. Machine Learning 39(2/3), 135–168 (2000)CrossRefzbMATHGoogle Scholar
  13. 13.
    Schietgat, L., Vens, C., Struyf, J., Blockeel, H., Kocev, D., Džeroski, S.: Predicting gene function using hierarchical multi-label decision tree ensembles. BMC bioinformatics 11(1), 2 (2010)CrossRefzbMATHGoogle Scholar
  14. 14.
    Shivakumar, P., Krauthammer, M.: Structural similarity assessment for drug sensitivity prediction in cancer. Bioinformatics 10, S17 (2009)Google Scholar
  15. 15.
    Su, H., Heinonen, M., Rousu, J.: Structured Output Prediction of Anti-Cancer Drug Activity. In: Dijkstra, T.M.H., Tsivtsivadze, E., Marchiori, E., Heskes, T. (eds.) PRIB 2010. LNCS, vol. 6282, pp. 38–49. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  16. 16.
    Trotter, M., Buxton, M., Holden, S.: Drug design by machine learning: support vector machines for pharmaceutical data analysis. Comp. and Chem. 26, 1–20 (2001)CrossRefGoogle Scholar
  17. 17.
    Wang, Y., Bolton, E., Dracheva, S., Karapetyan, K., Shoemaker, B., Suzek, T., Wang, J., Xiao, J., Zhang, J., Bryant, S.: An overview of the pubchem bioassay resource. Nucleic Acids Research 38, D255–D266 (2009)CrossRefGoogle Scholar
  18. 18.
    Yan, R., Tesic, J., Smith, J.: Model-shared subspace boosting for multi-label classification. In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 834–843. ACM (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Hongyu Su
    • 1
  • Juho Rousu
    • 1
  1. 1.Department of Computer ScienceUniversity of HelsinkiFinland

Personalised recommendations