Abstract
Artificial neural networks are a computational system, and usually, backpropagation algorithm is used for learning a task, because of its simplicity. However, backpropagation algorithm is likely to converge to a local minimum or saddle point, so that a global minimum may not be found. Differential Evolution (DE) is a simple yet powerful global optimization algorithm for solving multi-dimensional continuous functions. In this paper, we propose a new DE algorithm by combining two excellent DE algorithms, Adaptive Cauchy DE (ACDE) and Self-adaptive DE (SaDE). ACDE shows promising performance by using the Cauchy distribution based on control parameter adaptation. However, ACDE uses only one mutation strategy. SaDE adapts mutation strategies automatically, which shows its effectiveness. Therefore, we extend ACDE with the strategy adaptation of SaDE for enhancing the global optimization performance. The result indicates that the extended ACDE performs better than standard DE not only on conventional benchmark problems but also for training neural networks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Storn, R., Price, K.: Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim. 11(4), 341–359 (1997)
Price, K., Storn, R.M., Lampinen, J.A.: Differential Evolution: A Practical Approach to Global Optimization. Springer Science & Business Media, New York (2006)
Ilonen, J., Kamarainen, J.-K., Lampinen, J.: Differential evolution training algorithm for feed-forward neural networks. Neural Process. Lett. 17(1), 93–105 (2003)
Slowik, A., Bialko, M.: Training of artificial neural networks using differential evolution algorithm. In: 2008 Conference on Human System Interactions, pp. 60–65. IEEE (2008)
Wang, L., Zeng, Y., Chen, T.: Back propagation neural network with adaptive differential evolution algorithm for time series forecasting. Expert Syst. Appl. 42(2), 855–863 (2015)
Donate, J.P., Li, X., Sánchez, G.G., de Miguel, A.S.: Time series forecasting by evolving artificial neural networks with genetic algorithms, differential evolution and estimation of distribution algorithm. Neural Comput. Appl. 22(1), 11–20 (2013)
Piotrowski, A.P.: Differential evolution algorithms applied to neural network training suffer from stagnation. Appl. Soft Comput. 21, 382–406 (2014)
Choi, T.J., Ahn, C.W., An, J.: An adaptive cauchy differential evolution algorithm for global numerical optimization. Sci. World J. 2013, 1–12 (2013). https://doi.org/10.1155/2013/969734. Article no. 969734
Qin, A.K., Huang, V.L., Suganthan, P.N.: Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE Trans. Evol. Comput. 13(2), 398–417 (2009)
Zhang, X., Tian, Y., Cheng, R., Jin, Y.: A decision variable clustering-based evolutionary algorithm for large-scale many-objective optimization. IEEE Trans. Evol. Comput. PP(99), 1 (2016)
Cheng, R., Jin, Y., Narukawa, K., Sendhoff, B.: A multiobjective evolutionary algorithm using gaussian process-based inverse modeling. IEEE Trans. Evol. Comput. 19(6), 838–856 (2015)
Zhang, L., Pan, H., Su, Y., Zhang, X., Niu, Y.: A mixed representation-based multiobjective evolutionary algorithm for overlapping community detection. IEEE Trans. Cybern. 47, 2703–2716 (2017)
Pan, L., He, C., Tian, Y., Su, Y., Zhang, X.: A region division based diversity maintaining approach for many-objective optimization. Integr. Comput. Aided Eng. 24, 279–296 (2017)
He, C., Tian, Y., Jin, Y., Zhang, X., Pan, L.: A radial space division based evolutionary algorithm for many-objective optimization. Appl. Soft Comput. 61, 603–621 (2017)
Ma, X., Liu, F., Qi, Y., Wang, X., Li, L., Jiao, L., Yin, M., Gong, M.: A multiobjective evolutionary algorithm based on decision variable analyses for multiobjective optimization problems with large-scale variables. IEEE Trans. Evol. Comput. 20(2), 275–298 (2016)
Gong, M., Liu, J., Li, H., Cai, Q., Su, L.: A multiobjective sparse feature learning model for deep neural networks. IEEE Trans. Neural Netw. Learn. Syst. 26(12), 3263–3277 (2015)
Cheng, R., Jin, Y.: A competitive swarm optimizer for large scale optimization. IEEE Trans. Cybern. 45(2), 191–204 (2015)
Yao, X., Liu, Y., Lin, G.: Evolutionary programming made faster. IEEE Trans. Evol. Comput. 3(2), 82–102 (1999)
Lichman, M.: UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine, CA (2013). http://archive.ics.uci.edu/ml
Acknowledgement
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. NRF-2017R1C1B2012752). In addition, this work was supported by the NRF grant funded by MSIT (No. NRF-2015R1D1A1A02062017). Correspondence should be addressed to Dr. Chang Wook Ahn; cwan@gist.ac.kr.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Choi, T.J., Ahn, C.W. (2017). Adaptive Cauchy Differential Evolution with Strategy Adaptation and Its Application to Training Large-Scale Artificial Neural Networks. In: He, C., Mo, H., Pan, L., Zhao, Y. (eds) Bio-inspired Computing: Theories and Applications. BIC-TA 2017. Communications in Computer and Information Science, vol 791. Springer, Singapore. https://doi.org/10.1007/978-981-10-7179-9_39
Download citation
DOI: https://doi.org/10.1007/978-981-10-7179-9_39
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-7178-2
Online ISBN: 978-981-10-7179-9
eBook Packages: Computer ScienceComputer Science (R0)