Advertisement

Discovering Networks of Interdependent Features in High-Dimensional Problems

  • Michał Dramiński
  • Michał J. Da̧browski
  • Klev Diamanti
  • Jacek Koronacki
  • Jan KomorowskiEmail author
Chapter
First Online:
Part of the Studies in Big Data book series (SBD, volume 16)

Abstract

The availability of very large data sets in Life Sciences provided earlier by the technological breakthroughs such as microarrays and more recently by various forms of sequencing has created both challenges in analyzing these data as well as new opportunities. A promising, yet underdeveloped approach to Big Data, not limited to Life Sciences, is the use of feature selection and classification to discover interdependent features. Traditionally, classifiers have been developed for the best quality of supervised classification. In our experience, more often than not, rather than obtaining the best possible supervised classifier, the Life Scientist needs to know which features contribute best to classifying observations (objects, samples) into distinct classes and what the interdependencies between the features that describe the observation. Our underlying hypothesis is that the interdependent features and rule networks do not only reflect some syntactical properties of the data and classifiers but also may convey meaningful clues about true interactions in the modeled biological system. In this chapter we develop further our method of Monte Carlo Feature Selection and Interdependency Discovery (MCFS and MCFS-ID, respectively), which are particularly well suited for high-dimensional problems, i.e., those where each observation is described by very many features, often many more features than the number of observations. Such problems are abundant in Life Science applications. Specifically, we define Inter-Dependency Graphs (termed, somewhat confusingly, ID Graphs) that are directed graphs of interactions between features extracted by aggregation of information from the classification trees constructed by the MCFS algorithm. We then proceed with modeling interactions on a finer level with rule networks. We discuss some of the properties of the ID graphs and make a first attempt at validating our hypothesis on a large gene expression data set for CD4\(^{+}\) T-cells. The MCFS-ID and ROSETTA including the Ciruvis approach offer a new methodology for analyzing Big Data from feature selection, through identification of feature interdependencies, to classification with rules according to decision classes, to construction of rule networks. Our preliminary results confirm that MCFS-ID is applicable to the identification of interacting features that are functionally relevant while rule networks offer a complementary picture with finer resolution of the interdependencies on the level of feature-value pairs.

Keywords

MCFS-ID ROSETTA Ciruvis High-dimensional problems Gene expression data 
This is a preview of subscription content, log in to check access.

References

  1. 1.
    Consortium, Encode Project, Bernstein et al: An integrated encyclopedia of DNA elements in the human genome. Nature 489(7414), 57–74 (2012). doi: 10.1038/nature11247 CrossRefGoogle Scholar
  2. 2.
    Birney, E., et al.: Identification and analysis of functional elements in 1% of the human genome by the ENCODE pilot project. Nature 447(7146), 799–816 (2007)CrossRefGoogle Scholar
  3. 3.
    Beck, T., Hastings, R.K., Gollapudi, S., Free, R.C., Brookes, A.J.: GWAS Central: a comprehensive resource for the comparison and interrogation of genome-wide association studies. Eur. J. Hum. Genet. 22(7), 949–952 (2014). doi: 10.1038/ejhg.2013.274 CrossRefGoogle Scholar
  4. 4.
    Bernstein, B.E., et al.: The NIH roadmap epigenomics mapping consortium. Nat. Biotechnol. 28(10), 1045–1048 (2010). doi: 10.1038/nbt1010-1045 CrossRefGoogle Scholar
  5. 5.
    Genomes Project, Consortium, Abecasis, G. R. et al: An integrated map of genetic variation from 1,092 human genomes. Nature 491(7422), 56–65 (2012). doi: 10.1038/nature11632 CrossRefGoogle Scholar
  6. 6.
    Dudoit, S., Fridlyand, J.: Classification in microarray experiments. In: Speed, T. (ed.) Statistical Analysis of Gene Expression Microarray Data, pp. 93–158. Chapman & Hall/CRC (2003)Google Scholar
  7. 7.
    Saeys, Y., Inza, I., Larrañaga, P.: A review of featrure selection techniques in bioinformatics. Bioinformatics 23(19), 2507–2517 (2007)CrossRefGoogle Scholar
  8. 8.
    Tibshirani, R., Hastie, T., Narasimhan, B., Chu, G.: Diagnosis of multiple cancer types by nearest shrunken centroids of gene exressions. Proc. Natl. Acad. Sci. USA 99, 6567–6572 (2002)CrossRefGoogle Scholar
  9. 9.
    Tibshirani, R., Hastie, T., Narasimhan, B., Chu, G.: Class prediction by nearest shrunken centroids, with applications to DNA microarrays. Statis. Sci. 18, 104–117 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Li, Y., Campbell, C., Tipping, M.: Bayesian automatic relevance determination algorithms for classifying gene expression data. Bioinformatics 18(10), 1332–1339 (2002)CrossRefGoogle Scholar
  11. 11.
    Lu, C., Devos, A., Suykens, J.A., Arús, C., Van Huffel, S.: Bagging linear sparse bayesian learning models for variable selection in cancer diagnosis. IEEE Trans. Inf. Technol. Biomed. 11, 338–347 (2007)CrossRefGoogle Scholar
  12. 12.
    Chrysostomou, K., Chen, Sherry Y., S.Y. and Liu, X.: Combining multiple classifiers for wrapper feature selection. Int. J. Data Mining Modell. Manag. 1, 91–102 (2008)Google Scholar
  13. 13.
    Breiman, L., Cutler, A.: Random forests—classification/clustering manual. http://www.math.usu.edu/~adele/forests/cc_home.htm (2008)
  14. 14.
    Diaz-Uriarte, R., de Andres, S.A.: Gene selection and classification of microarray data using random forest. BMC Bioinform. 7(3), (2006). doi: 10.1186/1471-2105-7-3 Google Scholar
  15. 15.
    Strobl, C., Boulesteix, A.-L., Zeileis, A., Hothorn, T.: Bias in random forest variable importance measures: illustrations, sources, and a solution. BMC Bioinform. 8(25), (2007). doi: 10.1186/1471-2105-8-25
  16. 16.
    Archer, K.J., Kimes, R.V.: Empirical characterization of random forest variable importance measures. Comp. Stat. Data Anal. 52(4), 2249–2260 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Nicodemus, K.K., Malley, J.D., Strobl, C., Ziegler, A.: The behaviour of random forest permutation-based variable importance measures under predictor correlation. BMC Bioinform. 11, 110 (2010)CrossRefGoogle Scholar
  18. 18.
    Strobl, C., Boulesteix, A.-L., Kneib, T., Augustin, T., Zeileis, A.: Conditional variable importance for random forests. BMC Bioinform. 9(307), (2008). doi: 10.1186/1471-2105-9-307
  19. 19.
    Paul, J., Dupont, P.: Inferring statistically significant features from random forests. Neurocomputing 150, 471–480 (2015)CrossRefGoogle Scholar
  20. 20.
    Huynh-Thu, V.A.A., Saeys, Y., Wehenkel, L., Geurts, P.: Statistical interpretation of machine learning-based feature importance scores for biomarker discovery. Bioinformatics 28(13), 1766–1774 (2012)CrossRefGoogle Scholar
  21. 21.
    Dramiński, M., Koronacki, J., Komorowski, J.: A study on Monte Carlo Gene screening. In: Intelligent Information Processing and Web Mining, pp. 349–356. Springer (2005)Google Scholar
  22. 22.
    Dramiński, M., Rada Iglesias, A., Enroth, S., Wadelius, C., Koronacki, J., Komorowski, J.: Monte Carlo feature selection for supervised classification. Bioinformatics 24(1), 110–117 (2008)CrossRefGoogle Scholar
  23. 23.
    Dramiński, M., Kierczak, M., Nowak-Brzezińska, A., Koronacki, J.: The Monte Carlo feature selection and interdependency discovery is practically unbiased. Control Cybern. 40(2), 199–211 (2011)zbMATHGoogle Scholar
  24. 24.
    Dramiński, M., Kierczak, M., Koronacki, J. and Komorowski, J.: Monte Carlo feature selection and interdependency discovery in supervised classification. In: Advances in Machine Learning, vol. 2, pp. 371–385. Springer (2010)Google Scholar
  25. 25.
    Kierczak, M., Ginalski, K., Dramiński, M., Koronacki, J., Rudnicki, W., Komorowski, J.: A rough set-based model of HIV-1 RT Resistome. Bioinformatics a. Biol. Insights 3, 109–127 (2009)Google Scholar
  26. 26.
    Kierczak, M., Dramiński, M., Koronacki, J., Komorowski, J.: Computational analysis of local molecular interaction networks underlying change of HIV-1 resistance to selected reverse transcriptase inhibitors. Bioinformatics a. Biol. Insights 4, 137–146 (2010)Google Scholar
  27. 27.
    Bornelöv, S., Marillet, S., Komorowski, J.: Ciruvis: a web-based tool for rule networks and interaction detection using rule-based classifiers. BMC Bioinform. 15, 139 (2014)Google Scholar
  28. 28.
    Hvidsten, T.R., Wilczyński, B., Kryshtafovych, A., Tiuryn, J., Komorowski, J., Fidelis, K.: Discovering regulatory binding-site modules using rule-based learning. Genome Res. 15(6), 856–866 (2005)CrossRefGoogle Scholar
  29. 29.
    Ho, T.K.: The random subspace method for constructing decision forests. IEEE Trans. Pattern Analysis Mach. Intell. 20(8), 832–844 (1998)Google Scholar
  30. 30.
    Gyenesei, A., Wagner, U., Barkow-Oesterreicher, S., Stolte, E., Schlapbach, R.: Mining co-regulated gene profiles for the detection of functional associations in gene expression data. Bioinformatics 23(15), 1927–1935 (2007)CrossRefGoogle Scholar
  31. 31.
    Hastie, T., Tibshirani, R., Botstein, D., Brown, P.: Supervised harvesting of expression trees. Genome Biol. 2(1), research0003.1-0003.12 (2001)Google Scholar
  32. 32.
    Smyth, G.K., Yang, Y.H., Speed, T.: Statistical issues in cDNA microarray data analysis. In: Brownstein, M.J., Khodursky, A.B. (eds.) Functional Genomics: Methods and Protocols. Methods in Molecular Biology, vol. 224, pp. 111–136. Humana Press (2003)Google Scholar
  33. 33.
    Pawlak, Z.: Information systems: theoretical foundations. Inform. Syst. 6(3), 205–218 (1981)CrossRefzbMATHGoogle Scholar
  34. 34.
    Krzywinski, M., Schein, J., Birol, İ., Connors, J., Gascoyne, R., Horsman, D., Jones, S.J., Marra, M.A.: Circos: an information aesthetic for comparative genomics. Genome Res. 19(9), 1639–1645 (2009)CrossRefGoogle Scholar
  35. 35.
    Ye, C.J., et al.: Intersection of population variation and autoimmunity genetics in human T cell activation. Science 345(6202), 1254665 (2014)CrossRefGoogle Scholar
  36. 36.
    Ames, R.S., et al.: Human urotensin-II is a potent vasoconstrictor and agonist for the orphan receptor GPR14. Nature 401(6750), 282–6 (1999). doi: 10.1038/45809 CrossRefGoogle Scholar
  37. 37.
    Lehner, U., et al.: Ligands and signaling of the G-protein-coupled receptor GPR14, expressed in human kidney cells. Cell. Physiol. Biochem. 20(1–4), 181–192 (2007)CrossRefGoogle Scholar
  38. 38.

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Michał Dramiński
    • 1
  • Michał J. Da̧browski
    • 1
  • Klev Diamanti
    • 2
  • Jacek Koronacki
    • 1
  • Jan Komorowski
    • 3
    Email author
  1. 1.Institute of Computer SciencePolish Acad. SciWarsawPoland
  2. 2.Department of Cell and Molecular BiologyUppsala UniversityUppsalaSweden
  3. 3.Department of Cell and Molecular BiologyUppsala University and Institute of Computer Science, Polish Acad. SciUppsalaSweden

Personalised recommendations

Cite chapter

Buy options