Abstract
A neural network’s topology greatly influences its generalization ability. Many approaches to topology optimization employ heuristics, for example genetic algorithms, oftentimes consuming immense computational resources. In this contribution, we present a genetic algorithm for network topology optimization which can be deployed effectively in low-resource settings. To this end, we utilize the TensorFlow framework for network training and operate with several techniques reducing the computational load. The genetic algorithm is subsequently applied to the MNIST image classification task in two different scenarios.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous systems (2015). https://www.tensorflow.org/tutorials/estimators/cnn
Assunção, F., et al.: DENSER: deep evolutionary network structured representation. CoRR abs/1801.01563 (2018). http://arxiv.org/abs/1801.01563
Assunção, F., Lourenço, N., Machado, P., Ribeiro, B.: Evolving the topology of large scale deep neural networks. In: Castelli, M., Sekanina, L., Zhang, M., Cagnoni, S., García-Sánchez, P. (eds.) EuroGP 2018. LNCS, vol. 10781, pp. 19–34. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-77553-1_2
Baldominos, A., Saez, Y., Isasi, P.: Evolutionary convolutional neural networks: an application to handwriting recognition. Neurocomputing 283, 38–52 (2018)
Ciresan, D.C., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3642–3649 (2012)
Ciresan, D.C., et al.: Deep, big, simple neural nets for handwritten digit recognition. Neural Comput. 22(12), 3207–3220 (2010)
Ciresan, D.C., et al.: Flexible, high performance convolutional neural networks for image classification. In: Walsh, T. (ed.) Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, pp. 1237–1242 (2011)
He, K., et al.: Deep residual learning for image recognition. CoRR abs/1512.03385 (2015). http://arxiv.org/abs/1512.03385
LeCun, et al.: The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist/. Accessed 21 Sept 2018
Liu, H., et al.: Hierarchical representations for efficient architecture search. CoRR abs/1711.00436 (2017). http://arxiv.org/abs/1711.00436
Ma, B., et al.: Autonomous deep learning: a genetic DCNN designer for image classification. CoRR abs/1807.00284 (2018). http://arxiv.org/abs/1807.00284
Miller, G., et al.: Designing neural networks using genetic algorithms. In: Proceedings of the 3rd International Conference on Genetic Algorithms, pp. 379–384. Morgan Kaufmann Publishers Inc., San Francisco (1989)
Mitschke, N., et al.: Gradient based evolution to optimize the structure of convolutional neural networks. In: Proceedings of the 25th IEEE Conference on Image Processing (ICIP), pp. 3438–3442 (2018)
Real, E., et al.: Regularized evolution for image classifier architecture search. CoRR abs/1802.01548 (2018). http://arxiv.org/abs/1802.01548
Schwartz, D., et al.: Exhaustive learning. Neural Comput. 2, 371–382 (1990)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Litzinger, S., Klos, A., Schiffmann, W. (2019). Compute-Efficient Neural Network Architecture Optimization by a Genetic Algorithm. In: Tetko, I., Kůrková, V., Karpov, P., Theis, F. (eds) Artificial Neural Networks and Machine Learning – ICANN 2019: Deep Learning. ICANN 2019. Lecture Notes in Computer Science(), vol 11728. Springer, Cham. https://doi.org/10.1007/978-3-030-30484-3_32
Download citation
DOI: https://doi.org/10.1007/978-3-030-30484-3_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30483-6
Online ISBN: 978-3-030-30484-3
eBook Packages: Computer ScienceComputer Science (R0)