Abstract
Deep convolutional neural networks have been widely used in many areas. Generally, a vast amount of data are required for deep neural networks training, since they have a large number of parameters. This paper devotes to develop a many-objective convolutional neural network (MaO-CNN) model, which can obtain better classification performance than a single-objective one without sufficient training data. The main contributions of this paper are listed as follows: firstly, we propose many-class detection error trade-off (MaDET) and develop a MaO-CNN model in MaDET space; secondly, a hybrid framework of many-objective evolutionary algorithm is proposed for MaO-CNN model training; thirdly, a encoding method is designed for parameters encoding and MaO-CNN evolving. Experimental results based on well-known MNIST and SVHN datasets show that the new proposed model can obtain better results than a conventional one with the same amount of training data.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bai, S.: Scene categorization through using objects represented by deep features. Int. J. Pattern Recogn. Artif. Intell. 31(9), 1–21 (2017)
Goh, H., Thome, N., Cord, M., Lim, J.H.: Learning deep hierarchical visual feature coding. IEEE Trans. Neural Netw. Learn. Syst. 25(12), 2212–2225 (2014)
Gong, M., Liu, J., Li, H., Cai, Q., Su, L.: A multiobjective sparse feature learning model for deep neural networks. IEEE Trans. Neural Netw. Learn. Syst. 26(12), 3263–3277 (2015)
Goodfellow, I., Bengio, Y., Courville, A.: Deep learning (2016). http://www.deeplearningbook.org. Book in preparation for MIT Press
Sosa Hernández, V.A., Schütze, O., Emmerich, M.: Hypervolume maximization via set based Newton’s method. In: Tantar, A.-A., et al. (eds.) EVOLVE - A Bridge between Probability, Set Oriented Numerics, and Evolutionary Computation V. AISC, vol. 288, pp. 15–28. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07494-8_2
Ke, Q., Zhang, J., Song, H., Wan, Y.: Big data analytics enabled by feature extraction based on partial independence. Neurocomputing 288, 3–10 (2018). Learning System in Real-time Machine Vision
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning. In: NIPS Workshop on Deep Learning and Unsupervised Feature Learning (2010)
Trivedi, A., Srinivasan, D., Sanyal, K., Ghosh, A.: A survey of multi-objective evolutionary algorithms based on decomposition. IEEE Trans. Evol. Comput. 21(3), 440–462 (2017)
Vedaldi, A., Lenc, K.: MatConvNet - convolutional neural networks for MATLAB. In: Proceeding of the ACM International Conference on Multimedia (2015)
Wang, H., Jiao, L., Yao, X.: Two\(\_\)Arch2: an improved two-archive algorithm for many-objective optimization. IEEE Trans. Evol. Comput. 19(4), 524–541 (2015)
Xia, C., Qi, F., Shi, G.: Bottom-up visual saliency estimation with deep autoencoder-based sparse reconstruction. IEEE Trans. Neural Netw. Learn. Syst. 27(6), 1227–1240 (2016)
Xia, Y., Zhang, B., Coenen, F.: Face occlusion detection using deep convolutional neural networks. Int. J. Pattern Recogn. Artif. Intell. 30(09), 1–24 (2016)
Zhang, Q., Li, H.: Moea/d: a multiobjective evolutionary algorithm based on decomposition. IEEE Trans. Evol. Comput. 11(6), 712–731 (2007)
Zhao, J., et al.: Multiobjective optimization of classifiers by means of 3D convex-hull-based evolutionary algorithms. Inf. Sci. 367–368, 80–104 (2016)
Zhao, J.: 3D fast convex-hull-based evolutionary multiobjective optimization algorithm. Appl. Soft Comput. 67, 322–336 (2018)
Zhao, J.: Multiobjective sparse ensemble learning by means of evolutionary algorithms. Decis. Support Syst. 111, 86–100 (2018)
Zhao, Z., Jiao, L., Zhao, J., Gu, J., Zhao, J.: Discriminant deep belief network for high-resolution SAR image classification. Pattern Recogn. 61, 686–701 (2017)
Acknowledgment
This work was partially supported by the National Key Research and Development Plan (No. 2016YFC0600908), the National Natural Science Foundation of China (No. U1610124, 61572505 and 61772530), and the National Natural Science Foundation of Jiangsu Province (No. BK20171192).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, H., Xia, S., Zhao, J., Zhu, D., Yao, R., Niu, Q. (2018). Pareto-Based Many-Objective Convolutional Neural Networks. In: Meng, X., Li, R., Wang, K., Niu, B., Wang, X., Zhao, G. (eds) Web Information Systems and Applications. WISA 2018. Lecture Notes in Computer Science(), vol 11242. Springer, Cham. https://doi.org/10.1007/978-3-030-02934-0_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-02934-0_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-02933-3
Online ISBN: 978-3-030-02934-0
eBook Packages: Computer ScienceComputer Science (R0)