Skip to main content
SpringerLink
Log in

A Conjugate Gradient-Based Efficient Algorithm for Training Single-Hidden-Layer Neural Networks

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2016)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9950))

Included in the following conference series:

  • 2495 Accesses

Abstract

A single hidden layer neural networks (SHLNNs) learning algorithm has been proposed which is called Extreme Learning Machine (ELM). It shows extremely faster than typical back propagation (BP) neural networks based on gradient descent method. However, it requires many more hidden neurons than BP neural networks to achieve assortive classification accuracy. This then leads more test time which plays an important role in practice. A novel learning algorithm (USA) for SHLNNs has been presented which updates the weights by using gradient method in the ELM framework. In this paper, we employ the conjugate gradient method to train the SHLNNs on the MNIST digit recognition problem. The simulated experiment demonstrates the better generalization and less required hidden neurons than the common ELM and USA.

J. Wang—This work was supported in part by the National Natural Science Foundation of China (No. 61305075), the China Postdoctoral Science Foundation (No. 2012M520624), the Natural Science Foundation of Shandong Province (No. ZR2013FQ004, ZR2013DM015), the Specialized Research Fund for the Doctoral Program of Higher Education of China (No. 20130133120014) and the Fundamental Research Funds for the Central Universities (No. 13CX05016A, 14CX05042A, 15CX05053A, 15CX08011A).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: theory and applications. Neurocomputing 70(123), 489–501 (2006)

    Article  Google Scholar 

  2. Serre, D.: Matrices: Theory and Applications. Springer, New York (2002)

    MATH  Google Scholar 

  3. Rao, C.R., Mitra, S.K.: Generalized Inverse of Matrices and Its Applications. Wiley, New York (1971)

    MATH  Google Scholar 

  4. Zhang, P., Wang, X., Gu, D., Zhao, S.: Extreme learning machine based on conjugate gradient. J. Comput. Appl. 35(10), 2757–2760 (2015)

    Google Scholar 

  5. Chorowski, J., Wang, J., Zurada, J.M.: Review and performance comparison of SVM- and ELM-based classifiers. Neurocomputing 128(5), 507–516 (2014)

    Article  Google Scholar 

  6. Bartlett, P.L.: The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network. IEEE Trans. Inf. Theor. 44(2), 525–536 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  7. Widrow, B., Greenblatt, A., Kim, Y., Park, D.: The no-prop algorithm: a new learning algorithm for multilayer neural networks. Neural Netw. 37, 182–188 (2013)

    Article  Google Scholar 

  8. Zhu, Q.Y., Qin, A.K., Suganthan, P.N., Huang, G.B.: Evolutionary extreme learning machine. Pattern Recogn. 38(10), 1759–1763 (2005)

    Article  MATH  Google Scholar 

  9. Yu, D., Deng, L.: Efficient and effective algorithms for training single-hidden-layer neural networks. Pattern Recogn. Lett. 33(5), 554–558 (2012)

    Article  MathSciNet  Google Scholar 

  10. Hornik, K.: Approximation capabilities of multilayer feedforward networks. Neural Netw. 4(2), 251–257 (1991)

    Article  MathSciNet  Google Scholar 

  11. Leshno, M., Lin, V.Y., Pinkus, A., Schocken, S.: Multilayer feedforward networks with a nonpolynomial activation function can approximate any function. Neural Netw. 6(6), 861–867 (1993)

    Article  Google Scholar 

  12. Huang, G.B., Babri, H.A.: Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions. IEEE Trans. Neural Netw. 9(1), 224–229 (1998)

    Article  Google Scholar 

  13. Huang, G.B.: Learning capability and storage capacity of two hidden-layer feedforward networks. IEEE Trans. Neural Netw. 14(2), 274–281 (2003)

    Article  Google Scholar 

  14. Fletcher, R., Reeves, C.M.: Function minimization by conjugate gradients. Comput. J. 7, 149–154 (1964)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Information & Control Engineering, China University of Petroleum, Qingdao, 266580, China

    Xiaoling Gong, Jian Wang, Yanjiang Wang & Jacek M. Zurada

  2. College of Science, China University of Petroleum, Qingdao, 266580, China

    Xiaoling Gong, Jian Wang, Yanjiang Wang & Jacek M. Zurada

  3. Department of Electrical and Computer Engineering, University of Louisville, Louisville, KY, 40292, USA

    Xiaoling Gong, Jian Wang, Yanjiang Wang & Jacek M. Zurada

  4. Information Technology Institute, University of Social Sciences, 90-113, Łódź, Poland

    Xiaoling Gong, Jian Wang, Yanjiang Wang & Jacek M. Zurada

Authors
  1. Xiaoling Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanjiang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jacek M. Zurada
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoling Gong .

Editor information

Editors and Affiliations

  1. The University of Tokyo , Tokyo, Japan

    Akira Hirose

  2. Kobe University , Kobe, Japan

    Seiichi Ozawa

  3. Okinawa Institute of Science and Technology Graduate University, Onna, Japan

    Kenji Doya

  4. Nara Institute of Science and Technology , Ikoma, Japan

    Kazushi Ikeda

  5. Kyungpook National University , Daegu, Korea (Republic of)

    Minho Lee

  6. Chinese Academy of Sciences , Beijing, China

    Derong Liu

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Gong, X., Wang, J., Wang, Y., Zurada, J.M. (2016). A Conjugate Gradient-Based Efficient Algorithm for Training Single-Hidden-Layer Neural Networks. In: Hirose, A., Ozawa, S., Doya, K., Ikeda, K., Lee, M., Liu, D. (eds) Neural Information Processing. ICONIP 2016. Lecture Notes in Computer Science(), vol 9950. Springer, Cham. https://doi.org/10.1007/978-3-319-46681-1_56

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-46681-1_56

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-46680-4

  • Online ISBN: 978-3-319-46681-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation