Simplifying ConvNets for Fast Learning

  • Franck Mamalet
  • Christophe Garcia
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7553)


In this paper, we propose different strategies for simplifying filters, used as feature extractors, to be learnt in convolutional neural networks (ConvNets) in order to modify the hypothesis space, and to speed-up learning and processing times. We study two kinds of filters that are known to be computationally efficient in feed-forward processing: fused convolution/sub-sampling filters, and separable filters. We compare the complexity of the back-propagation algorithm on ConvNets based on these different kinds of filters. We show that using these filters allows to reach the same level of recognition performance as with classical ConvNets for handwritten digit recognition, up to 3.3 times faster.


Speedup Factor Convolutional Neural Network Kernel Size Hypothesis Space Handwritten Digit 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. of the IEEE (November 1998)Google Scholar
  2. 2.
    Chellapilla, K., Puri, S., Simard, P.: High Performance Convolutional Neural Networks for Document Processing. In: Proc. of the Int. Workshop on Frontiers in Handwriting Recognition, IWFHR 2006 (2006)Google Scholar
  3. 3.
    Garcia, C., Delakis, M.: Convolutional Face Finder: a neural architecture for fast and robust face detection. IEEE Trans. on Pattern Analysis and Machine Intelligence (November 2004)Google Scholar
  4. 4.
    Osadchy, M., LeCun, Y., Miller, M.L., Perona, P.: Synergistic face detection and pose estimation with energy-based model. In: Proc. of Advances in Neural Information Processing Systems, NIPS 2005 (2005)Google Scholar
  5. 5.
    Garcia, C., Duffner, S.: Facial image processing with convolutional neural networks. In: Proc. Int. Workshop on Advances in Pattern Recognition (2007)Google Scholar
  6. 6.
    Delakis, M., Garcia, C.: Text detection with Convolutional Neural Networks. In: Proc. of the Int. Conf. on Computer Vision Theory and Applications (2008)Google Scholar
  7. 7.
    Saidane, Z., Garcia, C.: Automatic scene text recognition using a convolutional neural network. In: Proc. of Int. Workshop on Camera-Based Document Analysis and Recognition (2007)Google Scholar
  8. 8.
    Hadsell, R., Sermanet, P., Scoffier, M., Erkan, A., Kavackuoglu, K., Muller, U., LeCun, Y.: Learning long-range vision for autonomous off-road driving. Journal of Field Robotics (February 2009)Google Scholar
  9. 9.
    Raiko, T., Valpola, H., LeCun, Y.: Deep learning made easier by linear transformations in perceptrons. In: Conf. on AI and Statistics (2012)Google Scholar
  10. 10.
    Reed, R.: Pruning algorithms - a survey. IEEE Trans. on Neural Networks (1993)Google Scholar
  11. 11.
    Jarrett, K., Kavukcuoglu, K., Ranzato, M., LeCun, Y.: What is the best multi-stage architecture for object recognition? In: Proc. Int. Conf. on Computer Vision (2009)Google Scholar
  12. 12.
    Mrazova, I., Kukacka, M.: Hybrid convolutional neural networks. In: Proc. of IEEE Int. Conf. on Industrial Informatics, INDIN 2008 (2008)Google Scholar
  13. 13.
    Holt, J., Baker, T.: Back propagation simulations using limited precision calculations. In: Proc. of Int. Joint Conf. on Neural Networks, IJCNN 1991 (1991)Google Scholar
  14. 14.
    Petrowski, A.: Choosing among several parallel implementations of the backpropagation algorithm. In: Proc. of IEEE Int. Conf. on Neural Networks (1994)Google Scholar
  15. 15.
    Ciresan, D., Meier, U., Gambardella, L.M., Schmidhuber, J.: Handwritten digit recognition with a committee of deep neural nets on GPUs. In: Computing Research Repository (2011)Google Scholar
  16. 16.
    Mamalet, F., Roux, S., Garcia, C.: Real-time video convolutional face finder on embedded platforms. EURASIP Journal on Embedded Systems (2007)Google Scholar
  17. 17.
    Mamalet, F., Roux, S., Garcia, C.: Embedded facial image processing with convolutional neural networks. In: Proc. of Int. Symp. on Circuits and Systems (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Franck Mamalet
    • 1
  • Christophe Garcia
    • 2
  1. 1.Orange LabsCesson-SévignéFrance
  2. 2.LIRIS, CNRS, Insa de LyonVilleurbanneFrance

Personalised recommendations