Abstract
The success of convolutional neural networks (CNNs) in the field of image recognition suggests that local connectivity is one of the key issues to exploit the prior information of structured data. But the problem of selecting optimal local receptive field still remains. We argue that the best way to select optimal local receptive field is to let CNNs learn how to choose it. To this end, we first use different sizes of local receptive fields to produce several sets of feature maps, then an element-wise max pooling layer is introduced to select the optimal neurons from these sets of feature maps. A novel training process ensures that each neuron of the model has the opportunity to be fully trained. The results of the experiments on handwritten Chinese character recognition show that the proposed method significantly improves the performance of traditional CNNs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Wu, C., Fan, W., He, Y., Sun, J., Naoi, S.: Cascaded Heterogeneous Convolutional Neural Networks for Handwritten Digit Recognition. In: 21st IEEE International Conference on Pattern Recognition, pp. 657–660. IEEE Press, Tsukuba (2012)
Ciresan, D., Schmidhuber, J.: Multi-Column Deep Neural Networks for Offline Handwritten Chinese Character Classification. Technical report, IDSIA (2013)
Yaniv, T., Ming, Y., Marc\’Aurelio, R., Lior, W.: DeepFace: Closing the Gap to Human-Level Performance in Face Verification. In: 27st IEEE Conference on Computer Vision and Pattern Recognition. IEEE Press, Columbus (2014)
Alexander, T., Christian, S.: DeepPose: Human Pose Estimation via Deep Neural Networks. In: 27th IEEE Conference on Computer Vision and Pattern Recognition. IEEE Press, Columbus (2014)
Alex, K., Ilya, S., Geoffrey, H.: ImageNet Classification with Deep Convolutional Neural Networks. In: Advances in Neural Information Processing Systems 25. NIPS Foundation, Nevada (2012)
Ossama, A., Li, D., Dong, Y.: Exploring Convolutional Neural Network Structures and Optimization Techniques for Speech Recognition. In: Interspeech 2013, ISCA (2013)
Yann, D., Yoshua, B.: Big Neural Networks Waste Capacity. In: International Conference on Learning Representations, Scottsdale (2013)
Coates, A., Ng, A., Lee, H.: An Analysis of Single-layer Networks in Unsupervised Feature Learning. In: 14th International Conference on Artificial Intelligence and Statistics, Reykjavik, pp. 215–223 (2011)
Coates, A., Ng, A.: Selecting Receptive Fields in Deep Networks. In: Advances in Neural Information Processing Systems 24. NIPS Foundation, Granada (2011)
Jia, Y., Huang, C., Darrell, T.: Beyond Spatial Pyramids: Receptive Field Learning for Pooled Image Features. In: 25th IEEE Conference on Computer Vision and Pattern Recognition. IEEE Press (2012)
Kong, S., Jiang, Z., Yang, Q.: Collaborative Receptive Field Learning. arXiv Preprint arXiv:1402.0170 (2014)
Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.R.: Improving Neural Networks by Preventing Co-adaptation of Feature Detectors. arXiv preprint arXiv:1207.0580 (2012)
Wan, L., Zeiler, M., Zhang, S., Cun, Y.L., Fergus, R.: Regularization of Neural Networks using Dropconnect. In: Proceedings of the 30th International Conference on Machine Learning, pp. 1058–1066 (2013)
Zeiler, M.D., Fergus, R.: Stochastic Pooling for Regularization of Deep Convolutional Neural Networks. In: International Conference on Learning Representations, Scottsdale (2013)
Goodfellow, I.J., Warde-Farley, D., Mirza, M., Courville, A., Bengio, Y.: Maxout Networks. In: International Conference on Learning Representations, Scottsdale (2013)
Wu, C., Fan, W., He, Y., Sun, J., Naoi, S.: Handwritten Character Recognition by Alternately Trained Relaxation Convolutional Neural Network. Submitted to ICFHR (2014)
Liu, C.L., Yin, F., Wang, D.H., Wang, Q.F.: CASIA Online and Offline Chinese Handwriting Databases. In: 2011 International Conference on Document Analysis and Recognition, pp. 37–41. IEEE Press (2011)
cuda-convnet project, https://code.google.com/p/cuda-convnet/
Simard, P., Steinkraus, D., Platt, J.C.: Best Practice for Convolutional Neural Networks Applied to Visual Document Analysis. In: 2003 International Conference on Document Analysis and Recognition. IEEE Press (2003)
Saxe, A., Koh, P.W., Chen, Z., Bhand, M., Suresh, B., Ng, A.Y.: On Random Weights and Unsupervised Feature Learning. In: Proceedings of the 28th International Conference on Machine Learning, pp. 1089–1096 (2010)
Bengio, Y.: Deep learning of representations: Looking forward. In: Dediu, A.-H., MartÃn-Vide, C., Mitkov, R., Truthe, B. (eds.) SLSP 2013. LNCS, vol. 7978, pp. 1–37. Springer, Heidelberg (2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chen, L., Wu, C., Fan, W., Sun, J., Naoi, S. (2014). Adaptive Local Receptive Field Convolutional Neural Networks for Handwritten Chinese Character Recognition. In: Li, S., Liu, C., Wang, Y. (eds) Pattern Recognition. CCPR 2014. Communications in Computer and Information Science, vol 484. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-45643-9_48
Download citation
DOI: https://doi.org/10.1007/978-3-662-45643-9_48
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-45642-2
Online ISBN: 978-3-662-45643-9
eBook Packages: Computer ScienceComputer Science (R0)