Skip to main content
SpringerLink
Log in

New Multi-Objective Algorithms for Neural Network Training Applied to Genomic Classification Data

  • Chapter
Foundations of Computational, Intelligence Volume 1

Part of the book series: Studies in Computational Intelligence ((SCI,volume 201))

  • 1010 Accesses

Abstract

One of the most used Artificial Neural Networks models is the Multi-Layer Perceptron, which is capable to fit any function as long as they have enough number of neurons and network layers. The process of obtaining a properly trained Artificial Neural Network usually requires a great effort in determining the parameters that will make it to learn. Currently there are a variety of algorithms for Artificial Neural Networks’s training working, simply, in order to minimize the sum of mean square error. However, even if the network reaches the global minimum error, it does not imply that the model response is optimal. Basically, a network with large number of weights but with small amplitudes behaves as an underfitted model that gradually overfits data during training. Solutions that have been overfitting are unnecessary complexity solutions.Moreover, solutions with low norm of the weights are those that present underfitting, with low complexity. The Multi-Objective Algorithmcontrols the weights amplitude by optimizing two objective functions: the error function and norm function. The high generalization capability of the Multi-Objective Algorithm and an automatic weight selection is aggregated by the LASSO approach, which generates networks with reduced number of weights when compared with Multi-Objective Algorithm solutions. Four data sets were chosen in order to compare and evaluate MOBJ, LASSO and Early-Stopping solutions. One generated from a function and tree available from a Machine Learning Repository. Additionally, the MOBJ and LASSO algorithms are applied to a microarray data set, which samples correspond to a genetic expression profile from DNA microarray technology of neoadjuvant chemotherapy (treatment given prior to surgery) for patients with breast cancer. Originally, the dataset is composed of 133 samples with 22283 attributes. By applying e probe section method described in the literature, 30 attributes were selected and used to train the Artificial Neural Networks. In average, the MOBJ and LASSO solutions were the same, the main difference is the simplified topology achieve by LASSO training method.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Medeiros, T., Braga, A.P.: A new decision strategy in multi-objective training of artificial neural networks. In: European Symposium on Neural Networks (ESANN 2007), pp. 555–560 (2007)

    Google Scholar 

  2. Kokshenev, I., Braga, A.P.: Complexity bounds for radial basis functions and multi- objective learning. In: European Symposium on Neural Networks (ESANN 2007), pp. 73–78 (2007)

    Google Scholar 

  3. Natowicz, R., Incitti, R., Horta, E.G., Charles, B., Guinot, P., Yan, K., Coutant, C., Andre, F., Pusztai, L., Rouzier, R.: Prediction of the outcome of preoperative chemotherapy in breast cancer by DNA probes that convey information on both complete and non complete responses. BMC Bioinformatics 9, 149 (2008)

    Article  Google Scholar 

  4. Simon, R.: Development and validation of therapeutically relevant multi-gene biomarker classifiers. J. Natl. Cancer Inst. 97, 866–867 (2005)

    Article  Google Scholar 

  5. Maglietta, R., D’Addabbo, A., Piepoli, A., Perri, F., Liuni, S., Pesole, G., Ancona, N.: Selection of relevant genes in cancer diagnosis based on their prediction accuracy. Artif. Intell. Med. 40, 29–44 (2007)

    Article  Google Scholar 

  6. Verducci, J.S., Melfi, V.F., Lin, S., Wang, Z., Roy, S., Sen, C.K.: Microarray analysis of gene expression: Considerations in data mining and statistical treatment. Physiol. Genomics 25, 355–363 (2006)

    Article  Google Scholar 

  7. Ancona, N., Maglietta, R., Piepoli, A., D’Addabbo, A., Cotugno, R., Savino, M., Liuni, S., Carella, M., Pesole, G., Perri, F.: On the statistical assessment of classifiers using DNA microarray data. BMC Bioinformatics 7, 387 (2006)

    Article  Google Scholar 

  8. Chen, J.J., Wang, S.J., Tsai, C.A., Lin, C.J.: Selection of differentially expressed genes in microarray data analysis. Pharmacogenomics 7, 212–220 (2007)

    Article  Google Scholar 

  9. Tozlu, S., Girault, I., Vacher, S., Vendrell, J., Andrieu, C., Spyratos, F., Cohen, P., Lidereau, R., Bieche, I.: Identification of novel genes that co-cluster with estrogen receptor alpha in breast tumor biopsy specimens, using a large-scale real-time reverse transcription-PCR approach. Endocr. Relat. Cancer 13, 1109–1120 (2006)

    Article  Google Scholar 

  10. Rouzier, R., Pusztai, L., Delaloge, S., Gonzalez-Angulo, A.M., Andre, F., Hess, K.R., Buzdar, A.U., Garbay, J.R., Spielmann, M., Mathieu, M.C., Symmans, W.F., Wagner, P., Atallah, D., Valero, V., Berry, D.A., Hortobagyi, G.N.: Nomograms to predict pathologic complete response and metastasis-free survival after preoperative chemotherapy for breast cancer. J. Clin. Oncol. 23, 8331–8339 (2005)

    Article  Google Scholar 

  11. Rouzier, R., Rajan, R., Wagner, P., Hess, K.R., Gold, D.L., Stec, J., Ayers, M., Ross, J.S., Zhang, P., Buchholz, T.A., Kuerer, H., Green, M., Arun, B., Hortobagyi, G.N., Symmans, W.F., Pusztai, L.: Microtubule-associated protein tau: A marker of paclitaxel sensitivity in breast cancer. Proc. Natl. Acad. Sci. USA 102, 8315–8320 (2005)

    Article  Google Scholar 

  12. Chang, J.C., Hilsenbeck, S.G., Fuqua, S.A.: The promise of microarrays in the management and treatment of breast cancer. Breast Cancer Res. 7, 100–104 (2005)

    Article  Google Scholar 

  13. Parma, G.G., Menezes, B.R., Braga, A.P.: Sliding mode algorithm for training multi-layer neural network. IEEE Letters 38(1), 97–98 (1998)

    Article  Google Scholar 

  14. Hagan, M.T., Menhaj, M.B.: Training feedforward networks with Marquardt algorithm. IEEE Transactions on Neural Networks 5(6), 989–993 (1994)

    Article  Google Scholar 

  15. Riedmiller, M., Braun, H.: A direct adaptative method for faster backpropagation learning: The RPROP algorthm. In: Proceddings of the IEEE Intl. Conf. on Neural Networks, San Francisco, pp. 586–591 (1993)

    Google Scholar 

  16. Fahlman, S.E.: Faster-learning variations on backpropagation: an empirical study. In: Proceddings of a 1988 Connectionist Models Summer School, Pittsburg, pp. 38–51 (1988)

    Google Scholar 

  17. Hinton, G.E.: Connectionist learning procedures. Artificial Inteligence 40, 185–234 (1989)

    Article  Google Scholar 

  18. Rumelhart, D., Hinton, G., Williams, R.: Learning representations by back-propagating errors. Nature 323, 533–536 (1986)

    Article  Google Scholar 

  19. Haykin, S.: Neural networks: A comprehensive foundation. Macmillan, New York (1994)

    MATH  Google Scholar 

  20. Rumelhart, D., McClelland, J.: Parallel Distributed Processing, vol. 1. MIT Press, Cambridge (1986)

    Google Scholar 

  21. Bartlett, P.L.: For valid generalization, the size of the weights is more important than the size of the network. In: Proceedings of NIPS 9, pp. 134–140 (1997)

    Google Scholar 

  22. Prechelt, L.: Automatic early stopping using cross validation: quantifying the criteria. Neural Networks 11(4), 761–767 (1998)

    Article  Google Scholar 

  23. Krogh, A., Hertz, J.A.: A Simple Weight Decay Can Improve Generalization. In: Proceedings of NIPS, vol. 4, pp. 950–957 (1991)

    Google Scholar 

  24. Reed, R.: Pruning algorithms - a survey. IEEE Transactions on Neural Networks 4(5), 740–746 (1993)

    Article  Google Scholar 

  25. Teixeira, R.A., Braga, A.P., Takahashi, R.H.C., Saldanha, R.R.: Recent Advances in the MOBJ Algorithm for training Artificial Neural Networks. International Journal of Neural Systems 11, 265–270 (2001)

    Article  Google Scholar 

  26. Bland, R.G., Goldfarb, D., Todd, M.J.: The ellipsoid method: a survey. Operations Research 29(6), 1039–1091 (1981)

    Article  MATH  MathSciNet  Google Scholar 

  27. Costa, M.A., Braga, A.P., Menezes, B.R., Teixeira, R.A., Parma, G.G.: Training neural networls with a multi-objective sliding mode control algorithm. Neurocomputing 51, 467–473 (2003)

    Article  Google Scholar 

  28. Costa, M.A., Braga, A.P., Menezes, B.R.: Improved Generalization Learning with Sliding Mode Control and the Levenberg-Marquadt Algorithm. In: Proceedings of VII Brazilian Symposium on Neural Networks (2002)

    Google Scholar 

  29. Jin, Y., Okabe, T., Sendhoff, B.: Neural network regularization and ensembling using multi-objective algorithms. In: Proceedings of the 2004 IEEE Congress on Evolutionary Computation, pp. 1–8 (2004)

    Google Scholar 

  30. Fieldsend, J.E., Singh, S.: Optimizing forecast model complexity using multi-objective evolutionary algorithms. In: Applications of Multi-Objective Evolutionary Algorithms, pp. 675–700. World Scientific, Singapore (2004)

    Google Scholar 

  31. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference and Prediction. Springer Series in Statistics (2001)

    Google Scholar 

  32. Tibshirani, R.: Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society. Series B (Methodological) 58(1), 267–288 (1996)

    MATH  MathSciNet  Google Scholar 

  33. Costa, M.A., Braga, A.P., Menezes, B.R.: Improving neural networks generalization with new constructive and pruning methods. Journal of intelligent & Fuzzy Systems 13, 73–83 (2003)

    Google Scholar 

  34. Harrison, D., Rubinfeld, D.L.: Hedonic prices and the demand for clean air. J. of Eviron. Economics & Management 5, 81–102 (1978)

    MATH  Google Scholar 

  35. May, K.O.: A Set of Independent Necessary and Sufficient Conditions for Simple Majority Decision. Econometrica 20(4), 680–684 (1952)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Depto. de Estatística, Universidade Federal de Minas Gerais, Brazil

    Marcelo Costa

  2. Depto. Ciência da Computação, Universidade Federal de Lavras, Brazil

    Thiago Rodrigues

  3. Depto. Engenharia Eletrônica, Universidade Federal de Minas Gerais, Brazil

    Euler Horta, Antônio Braga & Carmen Pataro

  4. ESIEE-Paris, Université Paris-Est, France

    René Natowicz & Arben Çela

  5. Institut Mondor de Médecine Moléculaire, Créteil, France

    Roberto Incitti

  6. Hôpital Tenon, department of Gynecology, Paris, France

    Roman Rouzier

Authors
  1. Marcelo Costa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thiago Rodrigues
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Euler Horta
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Antônio Braga
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Carmen Pataro
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. René Natowicz
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Roberto Incitti
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Roman Rouzier
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Arben Çela
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. College of Business Administration, Quantitative and Information System Department, Kuwait University, P.O. Box 5486, 13055, Safat, Kuwait

    Aboul-Ella Hassanien

  2. Center of Excellence for Quantifiable, Quality of Service, Norwegian University of Science & Technology, O.S. Bragstads plass 2E, 7491, Trondheim, Norway

    Ajith Abraham

  3. Department of Computer and Telecommunications Engineering, University ofWestern Macedonia, Agios Dimitrios Park, 50 100, Kozani, Greece

    Athanasios V. Vasilakos

  4. Dept. Electrical and Computer Engineering, University of Alberta, T6J 2V4, Edmonton,Alberta, Canada

    Witold Pedrycz

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Costa, M. et al. (2009). New Multi-Objective Algorithms for Neural Network Training Applied to Genomic Classification Data. In: Hassanien, AE., Abraham, A., Vasilakos, A.V., Pedrycz, W. (eds) Foundations of Computational, Intelligence Volume 1. Studies in Computational Intelligence, vol 201. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-01082-8_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-01082-8_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-01081-1

  • Online ISBN: 978-3-642-01082-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation