Skip to main content
SpringerLink
Log in

A Comparative Analysis of Various Regularization Techniques to Solve Overfitting Problem in Artificial Neural Network

  • Conference paper
  • First Online:
Data Science and Analytics (REDSET 2017)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 799))

Abstract

Neural networks having a large number of parameters are considered as very effective machine learning tool. But as the number of parameters becomes large, the network becomes slow to use and the problem of overfitting arises. Various ways to prevent overfitting of model are further discussed here and a comparative study has been done for the same. The effects of various regularization methods on the performance of neural net models are observed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  2. Smirnov, E.A., Timoshenko, D.M., Andrianov, S.N.: Comparison of regularization methods for imagenet classification with deep convolutional neural networks. AASRI Procedia 6, 89–94 (2014)

    Article  Google Scholar 

  3. Lau, K., López, R., Oñate, E.: A neural networks approach to aerofoil noise prediction. In: International Centre Numerical Methods Engineering, vol. CIMNE No-3 (2009)

    Google Scholar 

  4. Wan, L., Zeiler, M., Zhang, S., LeCun, Y., Fergus, R.: Regularization of neural networks using dropconnect. In: ICML, no. 1, pp. 109–111 (2013)

    Google Scholar 

  5. Ng, A.: Feature selection, L1 vs. L2 regularization, and rotational invariance. In: Twenty-First International Conference Machine Learning - ICML 2004, p. 78 (2004)

    Google Scholar 

  6. Golik, P., Doetsch, P., Ney, H.: Cross-entropy vs. squared error training: a theoretical and experimental comparison. In: Interspeech, vol. 13 (2013)

    Google Scholar 

  7. Kingma, D., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  8. Lopez, R., Brooks, T.F., Pope, D.S., Marcolini, M.A.: Airfoil self-noise data set. UCI Machine Learning Repository (2008)

    Google Scholar 

  9. Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. In: Proceedings of IEEE, vol. 86, no. 11, pp. 2278–2324 (1998)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dr. SPM International Institute of Information Technology Naya Raipur, Naya Raipur, India

    Shrikant Gupta, Rajat Gupta & Muneendra Ojha

  2. Indian Institute of Information Technology Allahabad, Allahabad, India

    K. P. Singh

Authors
  1. Shrikant Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rajat Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Muneendra Ojha
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. K. P. Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shrikant Gupta .

Editor information

Editors and Affiliations

  1. University of Arkansas, Fayetteville, AR, USA

    Brajendra Panda

  2. GD Goenka University, Gurugram, Haryana, India

    Sudeep Sharma

  3. GD Goenka University, Gurugram, Haryana, India

    Nihar Ranjan Roy

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gupta, S., Gupta, R., Ojha, M., Singh, K.P. (2018). A Comparative Analysis of Various Regularization Techniques to Solve Overfitting Problem in Artificial Neural Network. In: Panda, B., Sharma, S., Roy, N. (eds) Data Science and Analytics. REDSET 2017. Communications in Computer and Information Science, vol 799. Springer, Singapore. https://doi.org/10.1007/978-981-10-8527-7_30

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-8527-7_30

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-8526-0

  • Online ISBN: 978-981-10-8527-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation