Journal of Real-Time Image Processing

, Volume 16, Issue 1, pp 5–14 | Cite as

Real-time vehicle type classification with deep convolutional neural networks

  • Xinchen Wang
  • Weiwei ZhangEmail author
  • Xuncheng Wu
  • Lingyun Xiao
  • Yubin Qian
  • Zhi Fang
Special Issue Paper


Vehicle type classification technology plays an important role in the intelligent transport systems nowadays. With the development of image processing, pattern recognition and deep learning, vehicle type classification technology based on deep learning has raised increasing concern. In the last few years, convolutional neural network, especially Faster Region-convolutional neural networks (Faster R-CNN) has shown great advantages in image classification and object detection. It has superiority to traditional machine learning methods by a large margin. In this paper, a vehicle type classification system based on deep learning is proposed. The system uses Faster R-CNN to solve the task. Experimental results show that the method is not only time-saving, but also has more robustness and higher accuracy. Aimed at cars and trucks, it reached 90.65 and 90.51% accuracy. At last, we test the system on an NVDIA Jetson TK1 board with 192 CUDA cores that is envisioned to be forerunner computational brain for computer vision, robotics and self-driving cars. Experimental results show that it costs around 0.354 s to detect an image and keeps high accurate rate with the network embedded on NVDIA Jetson TK1.


Convolutional neural network Vehicle type classification Deep learning Intelligent transportation system Object detection 



This work was supported in part by National Fund for Fundamental Research (No. 282017Y-5303), in part by the Fund of National Automobile Accident In-depth Investigation System (No. HT2016X-007), in part by National Natural Science Foundation of China (No. 51675324), in part by Training and funding Program of Shanghai College young teachers (No. ZZGCD15102), in part by Scientific Research Project of Shanghai University of Engineering Science (No. 2016-19) and in part by the Shanghai University of Engineering Science Innovation Fund for Graduate Students (No. 16KY0602).


  1. 1.
    Hsieh, J.W., Chen, L.C., Chen, D.Y. et al.: Vehicle make and model recognition using symmetrical SURF. In: 2013 10th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 472–477 (2013)Google Scholar
  2. 2.
    Dong, Z., Wu, Y., Pei, M. et al.: Vehicle type classification using a semisupervised convolutional neural network. In: IEEE Transactions on Intelligent Transportation Systems, pp. 2247–2256 (2015)Google Scholar
  3. 3.
    Lai, A.H. Fung, G.S., Yung, N.H.: Vehicle type classification from visual-based dimension estimation. In: Proceedings of the IEEE Intelligent Transportation Systems Conference, pp. 201–206 (2001)Google Scholar
  4. 4.
    Gupte, S., Masoud, O., Martin, R.F., et al.: Detection and classification of vehicles. IEEE Trans. Intell. Transp. Syst. 3(1), 37–47 (2002)CrossRefGoogle Scholar
  5. 5.
    Saravi, S., Edirisinghe, E.A.: Vehicle make and model recognition in CCTV footage. In: 2013 18th International Conference on Digital Signal Processing (DSP), pp. 1–6 (2013)Google Scholar
  6. 6.
    Foresti, G.L., Murino, V., Regazzoni, C.: Vehicle recognition and tracking from road image sequences. IEEE Trans. Veh. Technol. 48(1), 301–318 (1999)CrossRefGoogle Scholar
  7. 7.
    Jang, D.M., Turk, M.: Car-rec: a real time car recognition system. In: 2011 IEEE Workshop on Applications of Computer Vision (WACV), Kona, HI, USA, pp. 599–605 (2011)Google Scholar
  8. 8.
    Tong, B., Fan, B., Wu, F.: Convolutional neural networks with neural cascade classifier for pedestrian detection. In: Chinese Conference on Pattern Recognition 2016, pp. 243–257. Springer Nature Singapore Pte LtdGoogle Scholar
  9. 9.
    Tomè, D., Monti, F., Baroffo, L., et al.: Deep convolution neural networks for pedestrian detection. Signal Process. Image Commun. 47, 482–489 (2016)CrossRefGoogle Scholar
  10. 10.
    Sarfraz, S.M., Saeed, A., Khan, M.H. et al. Bayesian prior models for vehicle make and model recognition. In: Proceedings of the 7th International Conference on Frontiers of Information Technology, pp. 35:1–35:6. ACM, New York (2009)Google Scholar
  11. 11.
    Ramnath, K., Hsiao, E. et al.: Car make and model recognition using 3D curve alignment. In: Winter Conference on Applications of Computer Vision, pp. 285–292 (2014)Google Scholar
  12. 12.
    Alonso, D., Salgado, L. et al.: Robust vehicle detection through multidimensional classification for on board video based systems. In: IEEE International Conference on Image Processing, pp. 4: IV–321–IV–324 (2007)Google Scholar
  13. 13.
    Chang, W.C., Cho, C.W.: Online boosting for vehicle detection. IEEE Trans. Syst. Man Cybernet. Part B (Cybernetics) 40(3), 892–902 (2010)CrossRefGoogle Scholar
  14. 14.
    Zhang, F.: Car Detection and Vehicle Type Classification Based on Deep Learning. Jiangsu University, Jiangsu (2016)Google Scholar
  15. 15.
    Zhang, F., Xu, X, Qiao, Y.: Deep classification of vehicle makers and models: the effectiveness of pre-training and data enhancement. In: 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 231–236 (2015)Google Scholar
  16. 16.
    Gu, J., Wang, Z., Kuen, J. et al.: Recent Advances in Convolutional Neural Networks. arXiv preprint arXiv:1512.07108[cs.CV] (2016)
  17. 17.
    LeCun, Y., Boser, B., Denker, J.S., et al.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)CrossRefGoogle Scholar
  18. 18.
    Krizhevsky, A., Sutskever, I., Hinton, G.: Imagenet classification with deep convolutional neural networks. In: Neural Information Processing Systems (NIPS), pp. 1097–1105 (2012)Google Scholar
  19. 19.
    He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. In: European Conference on Computer Vision (ECCV) (2014)Google Scholar
  20. 20.
    Girshick, R: Fast R-CNN. In: Proceedings of IEEE International Conference on Computer Vision (ICCV) (2015)Google Scholar
  21. 21.
    Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Proceedings of Computer vision-ECCV 2014. Springer, pp 818–833 (2014)Google Scholar
  22. 22.
    Ren, S., He, K., Girshick, R. et al.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Proceedings of Advances in Neural Information Processing Systems, pp. 91–99 (2015)Google Scholar
  23. 23.
    Ren, S.: Efficient Object Detection with Feature Sharing. University of Science and Technology of China, Hefei (2016)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  • Xinchen Wang
    • 1
  • Weiwei Zhang
    • 1
    Email author
  • Xuncheng Wu
    • 1
  • Lingyun Xiao
    • 2
  • Yubin Qian
    • 1
  • Zhi Fang
    • 1
  1. 1.College of Automotive EngineeringShanghai University of Engineering ScienceShanghaiChina
  2. 2.China National Institution of StandardizationBeijingChina

Personalised recommendations