Gene Selection for Multiclass Prediction by Weighted Fisher Criterion

  • Jianhua Xuan
  • Yue Wang
  • Yibin Dong
  • Yuanjian Feng
  • Bin Wang
  • Javed Khan
  • Maria Bakay
  • Zuyi Wang
  • Lauren Pachman
  • Sara Winokur
  • Yi-Wen Chen
  • Robert Clarke
  • Eric Hoffman
Open Access
Research Article


Gene expression profiling has been widely used to study molecular signatures of many diseases and to develop molecular diagnostics for disease prediction. Gene selection, as an important step for improved diagnostics, screens tens of thousands of genes and identifies a small subset that discriminates between disease types. A two-step gene selection method is proposed to identify informative gene subsets for accurate classification of multiclass phenotypes. In the first step, individually discriminatory genes (IDGs) are identified by using one-dimensional weighted Fisher criterion (wFC). In the second step, jointly discriminatory genes (JDGs) are selected by sequential search methods, based on their joint class separability measured by multidimensional weighted Fisher criterion (wFC). The performance of the selected gene subsets for multiclass prediction is evaluated by artificial neural networks (ANNs) and/or support vector machines (SVMs). By applying the proposed IDG/JDG approach to two microarray studies, that is, small round blue cell tumors (SRBCTs) and muscular dystrophies (MDs), we successfully identified a much smaller yet efficient set of JDGs for diagnosing SRBCTs and MDs with high prediction accuracies (96.9% for SRBCTs and 92.3% for MDs, resp.). These experimental results demonstrated that the two-step gene selection method is able to identify a subset of highly discriminative genes for improved multiclass prediction.


Support Vector Machine Muscular Dystrophy Gene Selection Molecular Diagnostics Gene Subset 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Bittner M, Meltzer P, Chen Y, et al.: Molecular classification of cutaneous malignant melanoma by gene expression profiling. Nature 2000, 406(6795):536-540. 10.1038/35020115CrossRefGoogle Scholar
  2. 2.
    Golub TR, Slonim DK, Tamayo P, et al.: Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 1999, 286(5439):531-537. 10.1126/science.286.5439.531CrossRefGoogle Scholar
  3. 3.
    Shipp MA, Ross KN, Tamayo P, et al.: Diffuse large B-cell lymphoma outcome prediction by gene-expression profiling and supervised machine learning. Nature Medicine 2002, 8(1):68-74. 10.1038/nm0102-68CrossRefGoogle Scholar
  4. 4.
    Liotta L, Petricoin E: Molecular profiling of human cancer. Nature Reviews Genetics 2000, 1(1):48-56. 10.1038/35049567CrossRefGoogle Scholar
  5. 5.
    Jain AK, Duin RPW, Mao J: Statistical pattern recognition: a review. IEEE Transactions on Pattern Analysis and Machine Intelligence 2000, 22(1):4-37. 10.1109/34.824819CrossRefGoogle Scholar
  6. 6.
    Jain AK, Zongker D: Feature selection: evaluation, application, and small sample performance. IEEE Transactions on Pattern Analysis and Machine Intelligence 1997, 19(2):153-158. 10.1109/34.574797CrossRefGoogle Scholar
  7. 7.
    Raudys SJ, Jain AK: Small sample size effects in statistical pattern recognition: recommendations for practitioners. IEEE Transactions on Pattern Analysis and Machine Intelligence 1991, 13(3):252-264. 10.1109/34.75512CrossRefGoogle Scholar
  8. 8.
    Fukunaga K: Introduction to Statistical Pattern Recognition. 2nd edition. Academic Press, Boston, Mass, USA; 1990.MATHGoogle Scholar
  9. 9.
    Devijver PA, Kittler J: Pattern Recognition: A Statistical Approach. Prentice-Hall, Englewood Cliffs, NJ, USA; 1982.MATHGoogle Scholar
  10. 10.
    Pudil P, Novovicova J, Kittler J: Floating search methods in feature selection. Pattern Recognition Letters 1994, 15(11):1119-1125. 10.1016/0167-8655(94)90127-9CrossRefGoogle Scholar
  11. 11.
    Dudoit S, Fridlyand J, Speed TP: Comparison of discrimination methods for the classification of tumors using gene expression data. Journal of the American Statistical Association 2002, 97(457):77-87. 10.1198/016214502753479248CrossRefMathSciNetMATHGoogle Scholar
  12. 12.
    Khan J, Wei JS, Ringnér M, et al.: Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks. Nature Medicine 2001, 7(6):673-679. 10.1038/89044CrossRefGoogle Scholar
  13. 13.
    Li T, Zhang C, Ogihara M: A comparative study of feature selection and multiclass classfication methods for tissue classification based on gene expression. Bioinformatics 2004, 20(15):2429-2437. 10.1093/bioinformatics/bth267CrossRefGoogle Scholar
  14. 14.
    Tibshirani R, Hastie T, Narasimhan B, Chu G: Diagnosis of multiple cancer types by shrunken centroids of gene expression. Proceedings of the National Academy of Sciences of the United States of America 2002, 99(10):6567-6572. 10.1073/pnas.082099299CrossRefGoogle Scholar
  15. 15.
    Xiong M, Fang X, Zhao J: Biomarker identification by feature wrappers. Genome Research 2001, 11(11):1878-1887.Google Scholar
  16. 16.
    Loog M: Approximate Pairwise Accuracy Criteria for Multiclass Linear Dimension Reduction: Generalisations of the Fisher Criterion. Delft University Press, Delft, The Netherlands; 1999.Google Scholar
  17. 17.
    Loog M, Duin RPW, Haeb-Umbach R: Multiclass linear dimension reduction by weighted pairwise Fisher criteria. IEEE Transactions on Pattern Analysis and Machine Intelligence 2001, 23(7):762-766. 10.1109/34.935849CrossRefGoogle Scholar
  18. 18.
    Koop JC: Generalized inverse of a singular matrix. Nature 1963, 200: 716.CrossRefGoogle Scholar
  19. 19.
    Press WM, Flannery BP, Teukolsky SA, Vetterling WT: Numerical Recipes: The Art of Scientific Computing. Cambridge University Press, New York, NY, USA; 1986.Google Scholar
  20. 20.
    Narendra PM, Fukunaga K: A branch and bound algorithm for feature subset selection. IEEE Transactions on Computers 1977, 26(9):917-922.CrossRefMATHGoogle Scholar
  21. 21.
    Marill T, Green DM: On the effectiveness of receptors in cognition system. IEEE Transactions on Information Theory 1963, 9: 11-17. 10.1109/TIT.1963.1057810CrossRefGoogle Scholar
  22. 22.
    Whitney AW: A direct method of nonparametric measurement selection. IEEE Transactions on Computers 1971, 20(9):1100-1103.CrossRefMathSciNetMATHGoogle Scholar
  23. 23.
    Stearns SD: On selecting features for pattern classifiers. Proceedings of the 3rd International Conference on Pattern Recognition, Coronado, Calif, USA, November 1976 71-75.Google Scholar
  24. 24.
    Haykin S: Neural Networks: A Comprehensive Foundation. 2nd edition. Prentice-Hall, Upper Saddle River, NJ, USA; 1999.MATHGoogle Scholar
  25. 25.
    Lee Y, Lee C-K: Classification of multiple cancer types by multicategory support vector machines using gene expression data. Bioinformatics 2003, 19(9):1132-1139. 10.1093/bioinformatics/btg102CrossRefGoogle Scholar
  26. 26.
    Ramaswamy S, Tamayo P, Rifkin R, et al.: Multiclass cancer diagnosis using tumor gene expression signatures. Proceedings of the National Academy of Sciences of the United States of America 2001, 98(26):15149-15154. 10.1073/pnas.211566398CrossRefGoogle Scholar
  27. 27.
    Bakay M, Chen Y-W, Borup R, Zhao P, Nagaraju K, Hoffman E: Sources of variability and effect of experimental approach on expression profiling data interpretation. BMC Bioinformatics 2002, 3(1):4. 10.1186/1471-2105-3-4CrossRefGoogle Scholar
  28. 28.
    Bakay M, Wang Z, Melcon G, et al.: Nuclear envelope dystrophies show a transcriptional fingerprint suggesting disruption of Rb-MyoD pathways in muscle regeneration. Brain 2006, 129(4):996-1013. 10.1093/brain/awl023CrossRefGoogle Scholar
  29. 29.
    Affymetrix Technical Note: Statistical algorithms description document. Affymetrix 2002. []Google Scholar
  30. 30.
    Zhao P, Seo J, Wang Z, Wang Y, Shneiderman B, Hoffman E: In vivo filtering of in vitro expression data reveals MyoD targets. Comptes Rendus - Biologies 2003, 326(10-11):1049-1065. 10.1016/j.crvi.2003.09.035CrossRefGoogle Scholar
  31. 31.
    Zhao P, Hoffman E: Embryonic myogenesis pathways in muscle regeneration. Developmental Dynamics 2004, 229(2):380-392. 10.1002/dvdy.10457CrossRefGoogle Scholar
  32. 32.
    Winokur S, Chen Y-W, Masny PS, et al.: Expression profiling of FSHD muscle supports a defect in specific stages of myogenic differentiation. Human Molecular Genetics 2003, 12(22):2895-2907. 10.1093/hmg/ddg327CrossRefGoogle Scholar
  33. 33.
    Bakay M, Zhao P, Chen J, Hoffman E: A web-accessible complete transcriptome of normal human and DMD muscle. Neuromuscular Disorders 2002, 12(1):S125-S141.CrossRefGoogle Scholar
  34. 34.
    Chen Y-W, Zhao P, Borup R, Hoffman E: Expression profiling in the muscular dystrophies: identification of novel aspects of molecular pathophysiology. Journal of Cell Biology 2000, 151(6):1321-1336. 10.1083/jcb.151.6.1321CrossRefGoogle Scholar
  35. 35.
    Hoffman E, Brown RH Jr., Kunkel LM: Dystrophin: the protein product of the Duchenne muscular dystrophy locus. Cell 1987, 51(6):919-928. 10.1016/0092-8674(87)90579-4CrossRefGoogle Scholar
  36. 36.
    Koening M, Hoffman E, Bertelson CJ, Monaco AP, Feener C, Kunkel LM: Complete cloning of the Duchenne muscular dystrophy (DMD) cDNA and preliminary genomic organization of the DMD gene in normal and affected individuals. Cell 1987, 50(3):509-517. 10.1016/0092-8674(87)90504-6CrossRefGoogle Scholar
  37. 37.
    Zhao P, Iezzi S, Carver E, et al.: Slug is a novel downstream target of MyoD. Temporal profiling in muscle regeneration. Journal of Biological Chemistry 2002, 277(33):30091-30101. 10.1074/jbc.M202668200CrossRefGoogle Scholar
  38. 38.
    Fernandes RJ, Skiena SS: Microarray synthesis through multiple-use PCR primer design. Bioinformatics 2002, 18(1):S128-S135. 10.1093/bioinformatics/18.suppl_1.S128CrossRefGoogle Scholar
  39. 39.
    Jaeger J, Weichenhan D, Ivandic B, Spang R: Early diagnostic marker panel determination for microarray based clinical studies. Statistical Applications in Genetics and Molecular Biology 2005., 4(1, article 9):Google Scholar
  40. 40.
    Li W: How many genes are needed for early detection of breast cancer, based on gene expression patterns in peripheral blood cells? Breast Cancer Research 2005, 7(5):E5. 10.1186/bcr1295CrossRefGoogle Scholar
  41. 41.
    Glas AM, Floore A, Delahaye LJMJ, et al.: Converting a breast cancer microarray signature into a high-throughput diagnostic test. BMC Genomics 2006, 7: 278. 10.1186/1471-2164-7-278CrossRefGoogle Scholar
  42. 42.
    Duin RPW: Classifiers in almost empty spaces. Proceedings of the 15th International Conference on Pattern Recognition (ICPR '00), Barcelona, Spain, September 2000 2: 1-7.CrossRefGoogle Scholar
  43. 43.
    Raudys SJ: Evolution and generalization of a single neurone—I: single-layer perceptron as seven statistical classifiers. Neural Networks 1998, 11(2):283-296. 10.1016/S0893-6080(97)00135-4CrossRefGoogle Scholar
  44. 44.
    Raudys SJ: Evolution and generalization of a single neurone—II: complexity of statistical classifiers and sample size considerations. Neural Networks 1998, 11(2):297-313. 10.1016/S0893-6080(97)00136-6CrossRefGoogle Scholar
  45. 45.
    Raudys SJ, Duin RPW: Expected classification error of the Fisher linear classifier with pseudo-inverse covariance matrix. Pattern Recognition Letters 1998, 19(5-6):385-392. 10.1016/S0167-8655(98)00016-6CrossRefMATHGoogle Scholar
  46. 46.
    Vapnik VN: Statistical Learning Theory. John Wiley & Sons, New York, NY, USA; 1998.MATHGoogle Scholar
  47. 47.
    Oja E: Subspace Methods of Pattern Recognition. John Wiley & Sons, New York, NY, USA; 1984.Google Scholar

Copyright information

© Jianhua Xuan et al. 2007

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Authors and Affiliations

  • Jianhua Xuan
    • 1
  • Yue Wang
    • 1
  • Yibin Dong
    • 1
  • Yuanjian Feng
    • 1
  • Bin Wang
    • 1
  • Javed Khan
    • 2
  • Maria Bakay
    • 3
  • Zuyi Wang
    • 1
    • 3
  • Lauren Pachman
    • 4
  • Sara Winokur
    • 5
  • Yi-Wen Chen
    • 3
  • Robert Clarke
    • 6
  • Eric Hoffman
    • 3
  1. 1.Department of Electrical and Computer EngineeringVirginia Polytechnic Institute and State UniversityArlingtonUSA
  2. 2.Department of Pediatric OncologyNational Cancer InstituteGaithersburgUSA
  3. 3.Research Center for Genetic MedicineChildren's National Medical CenterWashingtonUSA
  4. 4.Disease Pathogenesis ProgramChildren's Memorial Research CenterChicagoUSA
  5. 5.Department of Biological ChemistryUniversity of CaliforniaIrvineUSA
  6. 6.Lombardi Cancer CenterGeorgetown UniversityWashingtonUSA

Personalised recommendations