Advertisement

GA-Based Filter Selection for Representation in Convolutional Neural Networks

  • Junbong Kim
  • Minki Lee
  • Jongeun Choi
  • Kisung SeoEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11132)

Abstract

One of the deep learning models, a convolutional neural network (CNN) has been very successful in a variety of computer vision tasks. Features of a CNN are automatically generated, however, they can be further optimized since they often require large scale parallel operations and there exist the possibility of overlapping redundant features. The aim of this paper is to use feature selection via evolutionary algorithms to remove the irrelevant deep features. This will minimize the computational complexity and the amount of overfitting while maintaining a good quality of representation. We demonstrate the improvement of the filter representation by performing experiments on three data sets of CIFAR10, metal surface defects, and variation of MNIST and by analyzing the classification performance as well as the variance of the filter.

Keywords

CNN Feature representation Filter optimization 

Notes

Acknowledgement

This work was supported by National Research Foundation of Korea Grant funded by the Korea government (NRF-2016R1D1A1A09919650). This work was also funded by the Korea Meteorological Administration Research and Develop- ment Program under Grant KMIPA(KMI2018-06710).

References

  1. 1.
    Do, T.T., Hoang, T., Tan, D.K.L., Cheung, N.M.: From selective deep convolutional features to compact binary representations for image retrieval. arXiv preprint arXiv:1802.02899 (2018)
  2. 2.
    Goldberg, D.: Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, Reading (1989). Google Scholar (2014)Google Scholar
  3. 3.
    Han, S., Mao, H., Dally, W.J.: Deep compression: compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149 (2015)
  4. 4.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  5. 5.
    Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: CVPR, vol. 1, p. 3 (2017)Google Scholar
  6. 6.
    Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., Keutzer, K.: Squeezenet: Alexnet-level accuracy with 50x fewer parameters and \(<\) 0.5 mb model size. arXiv preprint arXiv:1602.07360 (2016)
  7. 7.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  8. 8.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015)CrossRefGoogle Scholar
  9. 9.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  10. 10.
    Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016)
  11. 11.
    Lin, K., Lu, J., Chen, C.S., Zhou, J.: Learning compact binary descriptors with unsupervised deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1183–1192 (2016)Google Scholar
  12. 12.
    Miller, J.F., Thomson, P.: Cartesian genetic programming. In: Poli, R., Banzhaf, W., Langdon, W.B., Miller, J., Nordin, P., Fogarty, T.C. (eds.) EuroGP 2000. LNCS, vol. 1802, pp. 121–132. Springer, Heidelberg (2000).  https://doi.org/10.1007/978-3-540-46239-2_9CrossRefGoogle Scholar
  13. 13.
    Rikhtegar, A., Pooyan, M., Manzuri-Shalmani, M.T.: Genetic algorithm-optimised structure of convolutional neural network for face recognition applications. IET Comput. Vis. 10(6), 559–566 (2016)CrossRefGoogle Scholar
  14. 14.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  15. 15.
    Suganuma, M., Shirakawa, S., Nagao, T.: A genetic programming approach to designing convolutional neural network architectures. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 497–504. ACM (2017)Google Scholar
  16. 16.
    Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)Google Scholar
  17. 17.
    Wu, J., Leng, C., Wang, Y., Hu, Q., Cheng, J.: Quantized convolutional neural networks for mobile devices. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4820–4828 (2016)Google Scholar
  18. 18.
    Xie, L., Yuille, A.L.: Genetic CNN. In: ICCV, pp. 1388–1397 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Junbong Kim
    • 1
  • Minki Lee
    • 1
  • Jongeun Choi
    • 2
  • Kisung Seo
    • 1
    Email author
  1. 1.Department of Electronic EngineeringSeokyeong UniversitySeoulKorea
  2. 2.School of Mechanical EngineeringYonsei UniversitySeoulKorea

Personalised recommendations