Pareto-Based Many-Objective Convolutional Neural Networks
Deep convolutional neural networks have been widely used in many areas. Generally, a vast amount of data are required for deep neural networks training, since they have a large number of parameters. This paper devotes to develop a many-objective convolutional neural network (MaO-CNN) model, which can obtain better classification performance than a single-objective one without sufficient training data. The main contributions of this paper are listed as follows: firstly, we propose many-class detection error trade-off (MaDET) and develop a MaO-CNN model in MaDET space; secondly, a hybrid framework of many-objective evolutionary algorithm is proposed for MaO-CNN model training; thirdly, a encoding method is designed for parameters encoding and MaO-CNN evolving. Experimental results based on well-known MNIST and SVHN datasets show that the new proposed model can obtain better results than a conventional one with the same amount of training data.
KeywordsConvolutional neural networks Many-objective optimization Evolutionary algorithms
This work was partially supported by the National Key Research and Development Plan (No. 2016YFC0600908), the National Natural Science Foundation of China (No. U1610124, 61572505 and 61772530), and the National Natural Science Foundation of Jiangsu Province (No. BK20171192).
- 4.Goodfellow, I., Bengio, Y., Courville, A.: Deep learning (2016). http://www.deeplearningbook.org. Book in preparation for MIT Press
- 5.Sosa Hernández, V.A., Schütze, O., Emmerich, M.: Hypervolume maximization via set based Newton’s method. In: Tantar, A.-A., et al. (eds.) EVOLVE - A Bridge between Probability, Set Oriented Numerics, and Evolutionary Computation V. AISC, vol. 288, pp. 15–28. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07494-8_2CrossRefzbMATHGoogle Scholar
- 8.Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning. In: NIPS Workshop on Deep Learning and Unsupervised Feature Learning (2010)Google Scholar
- 9.Trivedi, A., Srinivasan, D., Sanyal, K., Ghosh, A.: A survey of multi-objective evolutionary algorithms based on decomposition. IEEE Trans. Evol. Comput. 21(3), 440–462 (2017)Google Scholar
- 10.Vedaldi, A., Lenc, K.: MatConvNet - convolutional neural networks for MATLAB. In: Proceeding of the ACM International Conference on Multimedia (2015)Google Scholar