Pareto-Based Many-Objective Convolutional Neural Networks

  • Hongjian Zhao
  • Shixiong XiaEmail author
  • Jiaqi Zhao
  • Dongjun Zhu
  • Rui Yao
  • Qiang Niu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11242)


Deep convolutional neural networks have been widely used in many areas. Generally, a vast amount of data are required for deep neural networks training, since they have a large number of parameters. This paper devotes to develop a many-objective convolutional neural network (MaO-CNN) model, which can obtain better classification performance than a single-objective one without sufficient training data. The main contributions of this paper are listed as follows: firstly, we propose many-class detection error trade-off (MaDET) and develop a MaO-CNN model in MaDET space; secondly, a hybrid framework of many-objective evolutionary algorithm is proposed for MaO-CNN model training; thirdly, a encoding method is designed for parameters encoding and MaO-CNN evolving. Experimental results based on well-known MNIST and SVHN datasets show that the new proposed model can obtain better results than a conventional one with the same amount of training data.


Convolutional neural networks Many-objective optimization Evolutionary algorithms 



This work was partially supported by the National Key Research and Development Plan (No. 2016YFC0600908), the National Natural Science Foundation of China (No. U1610124, 61572505 and 61772530), and the National Natural Science Foundation of Jiangsu Province (No. BK20171192).


  1. 1.
    Bai, S.: Scene categorization through using objects represented by deep features. Int. J. Pattern Recogn. Artif. Intell. 31(9), 1–21 (2017)CrossRefGoogle Scholar
  2. 2.
    Goh, H., Thome, N., Cord, M., Lim, J.H.: Learning deep hierarchical visual feature coding. IEEE Trans. Neural Netw. Learn. Syst. 25(12), 2212–2225 (2014)CrossRefGoogle Scholar
  3. 3.
    Gong, M., Liu, J., Li, H., Cai, Q., Su, L.: A multiobjective sparse feature learning model for deep neural networks. IEEE Trans. Neural Netw. Learn. Syst. 26(12), 3263–3277 (2015)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Goodfellow, I., Bengio, Y., Courville, A.: Deep learning (2016). Book in preparation for MIT Press
  5. 5.
    Sosa Hernández, V.A., Schütze, O., Emmerich, M.: Hypervolume maximization via set based Newton’s method. In: Tantar, A.-A., et al. (eds.) EVOLVE - A Bridge between Probability, Set Oriented Numerics, and Evolutionary Computation V. AISC, vol. 288, pp. 15–28. Springer, Cham (2014). Scholar
  6. 6.
    Ke, Q., Zhang, J., Song, H., Wan, Y.: Big data analytics enabled by feature extraction based on partial independence. Neurocomputing 288, 3–10 (2018). Learning System in Real-time Machine VisionCrossRefGoogle Scholar
  7. 7.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  8. 8.
    Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning. In: NIPS Workshop on Deep Learning and Unsupervised Feature Learning (2010)Google Scholar
  9. 9.
    Trivedi, A., Srinivasan, D., Sanyal, K., Ghosh, A.: A survey of multi-objective evolutionary algorithms based on decomposition. IEEE Trans. Evol. Comput. 21(3), 440–462 (2017)Google Scholar
  10. 10.
    Vedaldi, A., Lenc, K.: MatConvNet - convolutional neural networks for MATLAB. In: Proceeding of the ACM International Conference on Multimedia (2015)Google Scholar
  11. 11.
    Wang, H., Jiao, L., Yao, X.: Two\(\_\)Arch2: an improved two-archive algorithm for many-objective optimization. IEEE Trans. Evol. Comput. 19(4), 524–541 (2015)CrossRefGoogle Scholar
  12. 12.
    Xia, C., Qi, F., Shi, G.: Bottom-up visual saliency estimation with deep autoencoder-based sparse reconstruction. IEEE Trans. Neural Netw. Learn. Syst. 27(6), 1227–1240 (2016)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Xia, Y., Zhang, B., Coenen, F.: Face occlusion detection using deep convolutional neural networks. Int. J. Pattern Recogn. Artif. Intell. 30(09), 1–24 (2016)CrossRefGoogle Scholar
  14. 14.
    Zhang, Q., Li, H.: Moea/d: a multiobjective evolutionary algorithm based on decomposition. IEEE Trans. Evol. Comput. 11(6), 712–731 (2007)CrossRefGoogle Scholar
  15. 15.
    Zhao, J., et al.: Multiobjective optimization of classifiers by means of 3D convex-hull-based evolutionary algorithms. Inf. Sci. 367–368, 80–104 (2016)CrossRefGoogle Scholar
  16. 16.
    Zhao, J.: 3D fast convex-hull-based evolutionary multiobjective optimization algorithm. Appl. Soft Comput. 67, 322–336 (2018)CrossRefGoogle Scholar
  17. 17.
    Zhao, J.: Multiobjective sparse ensemble learning by means of evolutionary algorithms. Decis. Support Syst. 111, 86–100 (2018)CrossRefGoogle Scholar
  18. 18.
    Zhao, Z., Jiao, L., Zhao, J., Gu, J., Zhao, J.: Discriminant deep belief network for high-resolution SAR image classification. Pattern Recogn. 61, 686–701 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Hongjian Zhao
    • 1
  • Shixiong Xia
    • 1
    Email author
  • Jiaqi Zhao
    • 1
  • Dongjun Zhu
    • 1
  • Rui Yao
    • 1
  • Qiang Niu
    • 1
  1. 1.School of Computer Science and Technology, Mine Digitization Engineering Research Center of the Ministry of EducationChina University of Mining and TechnologyXuzhouChina

Personalised recommendations