Skip to main content
SpringerLink
Log in

PCA-ELM: A Robust and Pruned Extreme Learning Machine Approach Based on Principal Component Analysis

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

It is well-known that single-hidden-layer feedforward networks (SLFNs) with additive models are universal approximators. However the training of these models was slow until the birth of extreme learning machine (ELM) “Huang et al. Neurocomputing 70(1–3):489–501 (2006)” and its later improvements. Before ELM, the faster algorithms for efficiently training SLFNs were gradient based ones which need to be applied iteratively until a proper model is obtained. This slow convergence implies that SLFNs are not used as widely as they could be, even taking into consideration their overall good performances. The ELM allowed SLFNs to become a suitable option to classify a great number of patterns in a short time. Up to now, the hidden nodes were randomly initiated and tuned (though not in all approaches). This paper proposes a deterministic algorithm to initiate any hidden node with an additive activation function to be trained with ELM. Our algorithm uses the information retrieved from principal components analysis to fit the hidden nodes. This approach considerably decreases computational cost compared to later ELM improvements and overcomes their performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Asuncion A, Newman D (2007) UCI machine learning repository. http://www.ics.uci.edu/~mlearn/MLRepository.html. Accessed 8 Sept 2007

  2. Cao J, Lin Z, Huang G (2011) Composite function wavelet neural networks with differential evolution and extreme learning machine. Neural Process Lett 33(3): 251–265

    Article  Google Scholar 

  3. Chen L, Zhou L, Pung HK (2008) Universal approximation and qos violation application of extreme learning machine. Neural Process Lett 28(2): 81–95

    Article  Google Scholar 

  4. Dunn OJ (1961) Multiple comparisons among means. J Am Stat Assoc 56: 52–56

    Article  MathSciNet  MATH  Google Scholar 

  5. Feng G, Huang GB, Lin Q, Gay R (2009) Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Trans Neural Netw 20(8): 1352–1357

    Article  Google Scholar 

  6. Friedman M (1940) A comparison of alternative tests of significance for the problem of m rankings. Ann Math Stat 11(1): 86–92

    Article  Google Scholar 

  7. Hochberg Y, Tamhane A (1987) Multiple comparison procedures. Wiley, New York

    Book  MATH  Google Scholar 

  8. Huang GB, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70(16–18): 3056–3062

    Article  Google Scholar 

  9. Huang GB, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71(16–18): 3460–3468

    Article  Google Scholar 

  10. Huang GB, Chen L, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17:4

    Google Scholar 

  11. Huang GB, Li MB, Chen L, Siew CK (2008) Incremental extreme learning machine with fully complex hidden nodes. Neurocomputing 71(4–6): 576–583

    Article  Google Scholar 

  12. Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern B 42(2): 513–529

    Article  Google Scholar 

  13. Huang GB, Zhu Q, Siew C (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3): 489–501

    Article  Google Scholar 

  14. Huang GB, Zhu QY, Siew CK (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. IEEE Int Conf Neural Netw Conf Proc 2: 985–990

    Google Scholar 

  15. Jaeger H (2001) The “echo state” approach to analysing and training recurrent neural networks. GMD Rep 148: 1435–2702

    Google Scholar 

  16. Kim J, Shin H, Lee Y, Lee M (2007) Algorithm for classifying arrhythmia using extreme learning machine and principal component analysis. In: 29th Annual international conference of the IEEE, engineering in medicine and biology society, 2007. EMBS, New York, pp 3257–3260

  17. Mich Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A (2010) OP-ELM: optimally pruned extreme learning machine. IEEE Trans Neural Netw 21(1): 158–162

    Article  Google Scholar 

  18. Miche Y, Sorjamaa A, Lendasse A (2008) Op-elm: theory, experiments and a toolbox. In: Artificial neural networks—ICANN 2008, lecture notes in computer science, vol 5163. Springer, Berlin, pp 145–154

  19. Rong HJ, Ong YS, Tan AH, Zhu Z (2008) A fast pruned-extreme learning machine for classification problem. Neurocomputing 72(1–3): 359–366

    Article  Google Scholar 

  20. Sánchez-Monedero J, Gutiérrez PA, Fernández-Navarro F, Hervás-Martínez C (2011) Weighting efficient accuracy and minimum sensitivity for evolving multi-class classifiers. Neural Process Lett 34(2): 101–116

    Article  Google Scholar 

  21. Schlkopf B, Smola AJ, Müller KR (1999) Kernel principal component analysis. In: Advances in kernel methods: support vector learning. MIT Press, Cambridge, pp 327–352

  22. Storn R, Price K. (1997) Differential evolution: a fast and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11: 341–359

    Article  MathSciNet  MATH  Google Scholar 

  23. Vapnik VN (1999) The nature of statistical learning theory. Springer, Berlin

    Google Scholar 

  24. Zhang R, Huang GB, Sundararajan N, Saratchandran P (2007) Multicategory classification using an extreme learning machine for microarray gene expression cancer diagnosis. Comput Biol Bioinform IEEE/ACM Trans 4(3): 485–495

    Article  Google Scholar 

  25. Zhu QY, Qin A, Suganthan P, Huang GB (2005) Evolutionary extreme learning machine. Pattern Recognit 38(10): 1759–1763

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Informatics, University of Pinar del Río, Pinar del Río, Cuba

    A. Castaño

  2. Advanced Concepts Team-European Space Research and Technology Centre (ESTEC), European Space Agency (ESA), Noordwijk, The Netherlands

    F. Fernández-Navarro

  3. Department of Computer Science and Numerical Analysis, University of Córdoba, Córdoba, Spain

    C. Hervás-Martínez

Authors
  1. A. Castaño
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. F. Fernández-Navarro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. C. Hervás-Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to F. Fernández-Navarro.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Castaño, A., Fernández-Navarro, F. & Hervás-Martínez, C. PCA-ELM: A Robust and Pruned Extreme Learning Machine Approach Based on Principal Component Analysis. Neural Process Lett 37, 377–392 (2013). https://doi.org/10.1007/s11063-012-9253-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-012-9253-x

Keywords

Search

Navigation