Abstract
A re-occurring challenge in applying feed-forward neural networks to a new dataset is the need to manually tune the neural network topology. If one’s attention is restricted to fully-connected three-layer networks, then there is only the need to manually tune the number of neurons in the single hidden layer. In this paper, we present a novel Ant Colony Optimization (ACO) algorithm that optimizes neural network topology for a given dataset. Our algorithm is not restricted to three-layer networks, and can produce topologies that contain multiple hidden layers, and topologies that do not have full connectivity between successive layers. Our algorithm uses Backward Error Propagation (BP) as a subroutine, but it is possible, in general, to use any neural network learning algorithm within our ACO approach instead. We describe all the elements necessary to tackle our learning problem using ACO, and experimentally compare the classification performance of the optimized topologies produced by our ACO algorithm with the standard fully-connected three-layer network topology most-commonly used in the literature.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Boryczka, U., Kozak, J.: Ant Colony Decision Trees. In: International Conference on Computational Collective Intelligence, pp. 4373–4382. Springer, Berlin (2010)
Boryczka, U., Kozak, J.: An Adaptive Discretization in the ACDT Algorithm for Continuous Attributes. In: Jędrzejowicz, P., Nguyen, N.T., Hoang, K. (eds.) ICCCI 2011, Part II. LNCS, vol. 6923, pp. 475–484. Springer, Heidelberg (2011)
Liao, T., Socha, K., de Oca, M.M., Stuetzle, T., Dorigo, M.: Ant colony optimization for mixed-variable optimization problems. IEEE Transactions on Evolutionary Computation (to appear, 2014)
Liu, Y.P., Wu, M.G., Qian, J.X.: Evolving neural networks using the hybrid of ant colony optimization and bp algorithms. In: Wang, J., Yi, Z., Żurada, J.M., Lu, B.-L., Yin, H. (eds.) ISNN 2006. LNCS, vol. 3971, pp. 714–722. Springer, Heidelberg (2006)
Otero, F., Freitas, A., Johnson, C.: Handling continuous attributes in ant colony classification algorithms. In: IEEE Symposium on Computational Intelligence in Data Mining (CIDM 2009), pp. 225–231 (2009)
Otero, F., Freitas, A., Johnson, C.: A New Sequential Covering Strategy for Inducing Classification Rules with Ant Colony Algorithms. IEEE Transactions on Evolutionary Computation 17(1), 64–74 (2013)
Otero, F.E.B., Freitas, A.A., Johnson, C.G.: Inducing Decision Trees with an Ant Colony Optimization Algorithm. Applied Soft Computing 12(11), 3615–3626 (2012)
Parpinelli, R.S., Lopes, H.S., Freitas, A.A.: Data mining with an ant colony optimization algorithm. IEEE Transactions on Evolutionary Computation 6(4), 321–332 (2002)
Salama, K., Abdelbar, A., Freitas, A.: Multiple Pheromone Types and Other Extensions to the Ant-Miner Classification Rule Discovery Algorithm. Swarm Intelligence 5(3-4), 149–182 (2011)
Salama, K., Abdelbar, A., Otero, F., Freitas, A.: Utilizing multiple pheromones in an ant-based algorithm for continuous-attribute classification rule discovery. Applied Soft Computing 13(1), 667–675 (2013)
Salama, K., Freitas, A.: Clustering-based Bayesian Multi-net Classifier Construction with Ant Colony Optimization. In: IEEE Congress on Evolutionary Computation (IEEE CEC), pp. 3079–3086 (2013)
Salama, K.M., Freitas, A.A.: Extending the ABC-Miner Bayesian Classification Algorithm. In: Terrazas, G., Otero, F.E.B., Masegosa, A.D. (eds.) NICSO 2013. SCI, vol. 512, pp. 1–12. Springer, Heidelberg (2014)
Salama, K., Freitas, A.: Learning Bayesian Network Classifiers Using Ant Colony Optimization. Swarm Intelligence 7(2-3), 229–254 (2013)
Salama, K., Freitas, A.: Ant Colony Algorithms for Constructing Bayesian Multi-net Classifiers. Intelligent Data Analysis (accepted, 2014)
Socha, K., Blum, C.: Training feed-forward neural networks with ant colony optimization: An application to pattern classification. In: 5th International Conference on Hybrid Intelligent Systems (HIS 2005), pp. 233–238 (2005)
Socha, K., Blum, C.: An ant colony optimization algorithm for continuous optimization: Application to feed-forward neural network training. Neural Computing & Applications 16, 235–247 (2007)
Socha, K., Dorigo, M.: Ant colony optimization for continuous domains. European Journal of Operational Research 185, 1155–1173 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Salama, K., Abdelbar, A.M. (2014). A Novel Ant Colony Algorithm for Building Neural Network Topologies. In: Dorigo, M., et al. Swarm Intelligence. ANTS 2014. Lecture Notes in Computer Science, vol 8667. Springer, Cham. https://doi.org/10.1007/978-3-319-09952-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-09952-1_1
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-09951-4
Online ISBN: 978-3-319-09952-1
eBook Packages: Computer ScienceComputer Science (R0)