Abstract
In recent years, the advance of GPUs led to the development in neural networks and deep learning. However, it is difficult to find a good CNN architecture people desire. In the past, people had to manually find the CNN architecture, and this is quite time-consuming and labor-intensive. In this paper, we use a self-adaptive harmony search algorithm to find the optimized convolutional neural network architecture for image recognition. The system architecture is divided into two phases. In the first phase, we search for the most suitable layer length of a CNN. In the second phase, we fine-tune the architecture found in the first phase or a pre-trained architecture. In the experiments, three popular and well-known datasets are used to evaluate the proposed methods and the state-of-the-art CNN search methods. The experimental results show that our methods achieve competitive performances compared with the other methods on CIFAR-10, and have the best performance among all the methods on MNIST and Caltech-101.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Albelwi, S., Mahmood, A.: Automated optimal architecture of deep convolutional neural networks for image recognition. In: Proceedings of the 15th IEEE International Conference on Machine Learning and Applications, pp. 53–60 (2016)
Al-Hyari, A., Areibi, S.: Design space exploration of convolutional neural networks based on evolutionary algorithms. J. Comput. Vis. Imaging Syst. 3(1) (2017)
Bottou, L.: Large-scale machine learning with stochastic gradient descent. In: Proceedings of the 19th International Conference on Computational Statistics, pp. 177–186 (2010)
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167v3, pp. 1–11 (2015)
Khan, H.A.: DM-L based feature extraction and classifier ensemble for object recognition. J. Sig. Inf. Process. 9(2), 92–110 (2018)
Krogh, A., Hertz, J.A.: A simple weight decay can improve generalization. Adv. Neural. Inf. Process. Syst. 4, 950–957 (1992)
Lin, M., Chen, Q., Yan, S.: Network in network. arXiv:1312.1400v3, pp. 1–10 (2014)
Pham, H., Guan, M.Y., Zoph, B., Le, Q.V., Dean, J.: Efficient neural architecture search via parameter sharing. arXiv:1802.03268v2 (2018)
Rosa, G., Papa, J., Marana, A., Scheirer, W., Cox, D.: Fine-tuning convolutional neural networks using harmony search. In: Proceedings the 20th Iberoamerican Congress on Pattern Recognition, pp. 683–690 (2015)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556v6, pp. 1–14 (2015)
Singh, A., Kingsbury, N.: Efficient convolutional network learning using parametric log based dual-tree wavelet scatternet. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 1140–1147 (2017)
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)
Tanno, R., Arulkumaran, K., Alexander, D.C., Criminisi, A., Nori, A.: Adaptive neural trees. arXiv:1807.06699v2 (2018)
Wang, C.M., Huang, Y.F.: Self-adaptive harmony search algorithm for optimization. Expert Syst. Appl. 37(4), 2826–2837 (2010)
Young, S.R., Rose, D.C., Karnowski, T.P., Lim, S.H., Patton, R.M.: Optimizing deep learning hyper-parameters through an evolutionary algorithm. In: Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments, Article No. 4 (2015)
Zeiler, M.D.: ADADELTA: an adaptive learning rate method. arXiv:1212.5701v1 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Huang, YF., Liu, JS. (2020). Optimizing Convolutional Neural Network Architecture Using a Self-adaptive Harmony Search Algorithm. In: Liu, Y., Wang, L., Zhao, L., Yu, Z. (eds) Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery. ICNC-FSKD 2019. Advances in Intelligent Systems and Computing, vol 1074. Springer, Cham. https://doi.org/10.1007/978-3-030-32456-8_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-32456-8_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32455-1
Online ISBN: 978-3-030-32456-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)