Abstract
In the previous chapter, we explored how initialization of the parameters affects the outcome of the model. In this chapter, we will explore ways to address the above issues by
-
Implementing optimization algorithms— mini-batch gradient descent, momentum, RMSprop, and Adam, and check for their convergence.
-
\(\ell _2\)-regularization, dropout regularization, and batch normalization gradient checking.
-
How to adjust train/dev/test data sets and analyze bias/variance.
-
Use TensorFlow for deep learning.
In A.I., the holy grail was how do you generate internal representations.
Geoffrey Hinton
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Hessian is a square matrix of second-order partial derivatives.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Ghatak, A. (2019). Optimization. In: Deep Learning with R. Springer, Singapore. https://doi.org/10.1007/978-981-13-5850-0_5
Download citation
DOI: https://doi.org/10.1007/978-981-13-5850-0_5
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-5849-4
Online ISBN: 978-981-13-5850-0
eBook Packages: Computer ScienceComputer Science (R0)