Abstract
Convolutional neural network (CNN) is one of the research focuses in machine learning in the last few years. But as the continuous development of CNN in vision and speech, the number of parameters is also increasing, too. CNN, which has millions of parameters, makes the memory of the model very large, and this impedes its widespread especially in mobile device. Based on the observation above, we not only design a CNN pruning method, where we prune unimportant feature maps, but also propose a separability values based number confirmation method which can relatively determine the appropriate pruning number. Experimental results show that, in the cifar-10 dataset, feature maps in each convolutional layer can be pruned by at least 15.6%, up to 59.7%, and the pruning process will not cause any performance loss. We also proved that the confirmation method is effective by a large number of repeated experiments which gradually prune feature maps of each convolutional layer.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)
Lu, H., Li, Y., Chen, M., et al.: Brain intelligence: go beyond artificial intelligence (2017)
Zeiler, M.D., Fergus, R.: Visual. Understand. Convolution. Netw. 8689:818–833 (2014)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: International Conference on Neural Information Processing Systems. Curran Associates Inc., 1097–1105 (2012)
Sermanet, P., Eigen, D., Zhang, X., et al.: OverFeat: integrated recognition, localization and detection using convolutional networks. Eprint Arxiv (2013)
Chatfield, K., Simonyan, K., Vedaldi, A., et al.: Return of the devil in the details: delving deep into convolutional nets. Comput. Sci. (2014)
Li, Y., Lu, H., Li, J., et al.: Underwater image de-scattering and classification by deep neural network. Comput. Electr. Eng. 54, 68–77 (2016)
Xu, W., Xu, W., Yang, M., et al.: 3D Convolutional neural networks for human action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 221–231 (2012)
Zou, W.Y., Wang, X., Sun, M., et al.: Generic object detection with dense neural patterns and regionlets. Eprint Arxiv (2014)
Liu, J., Ren, T., Wang, Y., et al.: Object proposal on RGB-D images via elastic edge boxes. Neurocomputing (2016)
Guo, J, Ren, T., Bei, J., et al.: Salient object detection in RGB-D image based on saliency fusion and propagation. In: International Conference on Internet Multimedia Computing and Service. ACM, 59 (2015)
Ren, T., Liu, Y., Ju, R., et al.: How important is location information in saliency detection of natural images. Multimed. Tools Appl. 75(5), 2543–2564 (2016)
Ren, T., Qiu, Z., Liu, Y., et al.: Soft-assigned bag of features for object tracking. Multimed. Syst. 21(2), 189–205 (2015)
Razavian, A.S., Azizpour, H., Sullivan, J., et al.: CNN features off-the-shelf: an astounding baseline for recognition. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops. IEEE Computer Society. 512–519 (2014)
Taigman, Y., Yang, M., Marc, et al.: DeepFace: closing the gap to human-level performance in face verification. Comput. Vis. Patt. Recogn. IEEE, 1701–1708 (2014)
Lu, H., Li, B., Zhu, J., et al.: Wound intensity correction and segmentation with convolutional neural networks. Concurr. Computat. Pract. Exper. (2016)
Lu, H., Li, Y., Nakashima, S., et al.: Single image dehazing through improved atmospheric light estimation. Multimed. Tools Appl. 75(24), 17081–17096 (2016)
Han, S., Mao, H., Dally, W.J.: Deep compression: compressing deep neural networks with pruning, trained quantization and huffman coding. Fiber 56(4), 3–7 (2015)
Courbariaux, M., Bengio, Y.: BinaryNet: Training Deep Neural Networks with Weights and Activations Constrained to +1 or −1 (2016)
Sun, Y., Wang, X., Tang, X.: Sparsifying neural network connections for face recognition. In: IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, 4856–4864 (2016)
Li, H., Kadav, A., Durdanovic, I., et al.: Pruning Filters for Efficient ConvNets (2017)
Rui, T., Zou, J., Zhou, Y., et al.: Pedestrian detection based on multi-convolutional features by feature maps pruning. Multimed. Tools Appl. 1–11 (2017)
Rui, T., Zou, J., Zhou, Y., et al.: Convolutional neural network feature maps selection based on LDA. Multimed. Tools Appl. 1–15 (2017)
Acknowledgements
This work was supported in part by National Natural Science Foundation of China (GrantNo.61472444, 61472392).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this chapter
Cite this chapter
Zou, J., Rui, T., Zhou, Y., Yang, C., Zhang, S. (2018). ConvNets Pruning by Feature Maps Selection. In: Lu, H., Xu, X. (eds) Artificial Intelligence and Robotics. Studies in Computational Intelligence, vol 752. Springer, Cham. https://doi.org/10.1007/978-3-319-69877-9_27
Download citation
DOI: https://doi.org/10.1007/978-3-319-69877-9_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-69876-2
Online ISBN: 978-3-319-69877-9
eBook Packages: EngineeringEngineering (R0)