Abstract
This paper describes a new optimization technique to perform an embedded implementation of convolutional neural networks (CNN). In this case, only the inference of convolutional neural networks is discussed. As known that both pooling layer and strided convolution can be used to summarize the data. So, the proposed technique aims to replace only max pooling layers by a strided convolution layers using the same filter size and stride of the old pooling layers in order to reduce the model size and improve the accuracy of a CNN. Also, pooling layer is parameter less. However, convolution layer has weights and biases to optimize. Then, the CNN can learn how to summarize the data. By replacing max pooling layers with strided convolution layers enhance the CNN accuracy and reduce the model size. This technique is proposed in order to build a CNN accelerator for real time application and embedded implementation.
The proposed optimizations are applied on some state-of-the-art CNN models and the obtained results are compared with the original ones. The proposed optimization is demonstrated for reducing the memory occupation of the model and achieving accuracy enhancement. The proposed technique enables possibility of the implementation of the convolutional neural network models in embedded systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Fei-Fei, L., Iyer, A., Koch, C., Perona, P.: What do we perceive in a glance of a real-world scene? J. Vis. 7(1), 10 (2007)
Szegedy Google Inc. Wei Liu University of North Carolina, Chapel Hill Yangqing Jia Google Inc. Pierre Sermanet Google Inc. Scott Reed University of Michigan Dragomir Anguelov Google Inc. Dumitru Erhan Google Inc. Vincent Vanhoucke Google Inc. Andrew Rabinovich Google Inc. Going deeper with convolutions Christian
Karen Simonyan Andrew Zisserman Visual Geometry Group, Department of Engineering Science, University of Oxford. Very Deep Convolutional Networks for Large-Scale Image Recognition
Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., Keutzer, K.: SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 0.5 MB model size
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998)
Krizhevsky, A., Sutskever, I., Hinton, G.: ImageNet classification with deep convolutional neural networks. In: NIPS’2012 (2012)
LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)
Chen, T., Li, M., Li, Y., Lin, M., Wang, N., Wang, M., Xiao, T., Xu, B., Zhang, C., Zhang, Z.: Mxnet: a flexible and efficient machine learning library for heterogeneous distributed systems (2015). arXiv:1512.01274
Snoek, J., Larochelle, H., Adams, R.P.: Practical Bayesian optimization of machine learning algorithms. In: NIPS (2012)
Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Neurocomputing 10(2), 99–127 (2002)
Ludermir, T.B., Yamazaki, A., Zanchettin, C.: An optimization methodology for neural network weights and architectures. IEEE Trans. Neural Netw. 17(6), 1452–1459 (2006)
Bergstra, J., Bengio, Y.: An optimization methodology for neural network weights and architectures. JMLR (2012)
Albelwi, S., Mahmood, A.: Automated optimal architecture of deep convolutional neural networks for image recognition. In: 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA), December 2016, pp. 53–60. IEEE
Becherer, N., Pecarina, J., Nykl, S., Hopkinson, K.: Improving optimization of convolutional neural networks through parameter fine-tuning. Neural Comput. Appl., 1–11 (2017)
Kong, C., Lucey, S.: Take it in your stride: do we need striding in CNNs? (2017). arXiv preprint arXiv:1712.02502
Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., Keutzer, K.: Squeezenet: Alexnet-level accuracy with 50x fewer parameters and < 0.5 mb model size (2016). arXiv preprint arXiv:1602.07360
Gazzah, S., Mhalla, A., Essoukri Ben Amara, N.: Vehicle detection on a video traffic scene: review and new perspectives. In: 2016 7th International Conference on Sciences of Electronics, Technologies of Information and Telecommunications (SETIT), Hammamet, pp. 448–454 (2016). https://doi.org/10.1109/setit.2016.7939912
Dahmane, K., Amara, N.E.B., Duthon, P., Bernardin, F., Colomb, M., Chausse, F.: The Cerema pedestrian database: a specific database in adverse weather conditions to evaluate computer vision pedestrian detectors. In: 2016 7th International Conference on Sciences of Electronics, Technologies of Information and Telecommunications (SETIT), December 2016, pp. 472–477. IEEE
Trimech, I.H., Maalej, A., Amara, N.E.B.: 3D facial expression recognition using nonrigid CPD registration method. In: 2016 7th International Conference on Sciences of Electronics, Technologies of Information and Telecommunications (SETIT), December 2016, pp. 478–481. IEEE
Jaouedi, N., Boujnah, N., Htiwich, O., Bouhlel, M.S.: Human action recognition to human behavior analysis. In: 2016 7th International Conference on Sciences of Electronics, Technologies of Information and Telecommunications (SETIT), December 2016, pp. 263–266. IEEE
Jdira, M.B., Imen, J., Kaïs, O.: Study of speaker recognition system based on feed forward deep neural networks exploring text-dependent mode. In: 7th International Conference on Sciences of Electronics, Technologies of Information and Telecommunications (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Ayachi, R., Afif, M., Said, Y., Atri, M. (2020). Strided Convolution Instead of Max Pooling for Memory Efficiency of Convolutional Neural Networks. In: Bouhlel, M., Rovetta, S. (eds) Proceedings of the 8th International Conference on Sciences of Electronics, Technologies of Information and Telecommunications (SETIT’18), Vol.1. SETIT 2018. Smart Innovation, Systems and Technologies, vol 146. Springer, Cham. https://doi.org/10.1007/978-3-030-21005-2_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-21005-2_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-21004-5
Online ISBN: 978-3-030-21005-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)