Abstract
In this study we propose a novel metaheuristic algorithm; namely “Gamma-Levy Hybrid Metaheuristic with Conditional Evolution (GLHM-CE)”. The proposed algorithm is evaluated over 28 Blackbox Problems of CEC-2013, Special Session on Real-Parameter Optimization and compared with modern metaheuristic and evolutionary algorithms like SHADE, Co-DE, and JADE. The statistical results show that GLHM-CE successfully circumvents local minimas on high dimensional blackbox functions and has a fast convergence. GLHM-CE is then used to optimize the hyperparameters of a static Deep Q Neural Network evaluated on OpenAI Gym Cartpole problem. The results evaluated over a total episodal run of 5000 shows a better stability of the DQN when the hyperparameters are optimized by GLHM-CE.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bellman, R.: Dynamic Programming. Dover Publications (1957)
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., Zaremba, W.: Openai gym (2016). arxiv:1606.01540
cartpole v1. https://gym.openai.com/envs/CartPole-v1/
LECUN, Y.: The mnist database of handwritten digits. http://yann.lecun.com/exdb/mnist/, https://ci.nii.ac.jp/naid/10027939599/en/
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.A.: Playing atari with deep reinforcement learning. (2013). CoRR arXiv:1312.5602
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., Hassabis, D.: Human-level control through deep reinforcement learning. Nature 518, 529 EP (2015). https://doi.org/10.1038/nature14236
Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML’10, pp. 807–814. Omnipress, USA. http://dl.acm.org/citation.cfm?id=3104322.3104425 (2010)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Parallel distributed processing: explorations in the microstructure of cognition. Learning Internal Representations by Error Propagation, vol. 1, pp. 318–362. MIT Press, Cambridge. http://dl.acm.org/citation.cfm?id=104279.104293 (1986)
Shi, X., Tian, S., Yu, L., Li, L., Gao, S.: Prediction of soil adsorption coefficient based on deep recursive neural network. Autom. Control. Comput. Sci. 51(5), 321–330 (2017). https://doi.org/10.3103/S0146411617050066
Tanabe, R., Fukunaga, A.: Success-history based parameter adaptation for differential evolution. In: 2013 IEEE Congress on Evolutionary Computation, pp. 71–78 (2013). https://doi.org/10.1109/CEC.2013.6557555
Wang, W., Yang, J., Xiao, J., Li, S., Zhou, D.: Face recognition based on deep learning. In: Zu, Q., Hu, B., Gu, N., Seng, S. (eds.) Human Centered Computing, pp. 812–820. Springer International Publishing, Cham (2015)
Wang, Y., Cai, Z., Zhang, Q.: Differential evolution with composite trial vector generation strategies and control parameters. IEEE Trans. Evol. Comput. 15(1), 55–66 (2011). https://doi.org/10.1109/TEVC.2010.2087271
Yang, X.S., Deb, S.: Multiobjective cuckoo search for design optimization. Comput. Oper. Res. 40(6), 1616–1624 (2013). https://doi.org/10.1016/j.cor.2011.09.026
Yuan, Y., Mou, L., Lu, X.: Scene recognition by manifold regularized deep learning architecture. IEEE Trans. Neural Netw. Learn. Syst. 26(10), 2222–2233 (2015). https://doi.org/10.1109/TNNLS.2014.2359471
Zambrano-Bigiarini, M., Gonzalez-Fernandez, Y.: cec2013: benchmark functions for the special session and competition on real-parameter single objective optimization at CEC-2013 (2015). http://CRAN.R-project.org/package=cec2013. R package version 0.1-5
Zhang, J., Sanderson, A.C.: Jade: adaptive differential evolution with optional external archive. IEEE Trans. Evol. Comput. 13(5), 945–958 (2009). https://doi.org/10.1109/TEVC.2009.2014613
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Banerjee, A., Ghosh, D., Das, S. (2020). A Gamma-Levy Hybrid MetaHeuristic for HyperParameter Tuning of Deep Q Network. In: Das, A., Nayak, J., Naik, B., Pati, S., Pelusi, D. (eds) Computational Intelligence in Pattern Recognition. Advances in Intelligent Systems and Computing, vol 999. Springer, Singapore. https://doi.org/10.1007/978-981-13-9042-5_54
Download citation
DOI: https://doi.org/10.1007/978-981-13-9042-5_54
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-9041-8
Online ISBN: 978-981-13-9042-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)