A Novel Image Classification Method with CNN-XGBoost Model

  • Xudie Ren
  • Haonan Guo
  • Shenghong LiEmail author
  • Shilin Wang
  • Jianhua Li
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10431)


Image classification problem is one of most important research directions in image processing and has become the focus of research in many years due to its diversity and complexity of image information. In view of the existing image classification models’ failure to fully utilize the information of images, this paper proposes a novel image classification method of combining the Convolutional Neural Network (CNN) and eXtreme Gradient Boosting (XGBoost), which are two outstanding classifiers. The presented CNN-XGBoost model provides more precise output by integrating CNN as a trainable feature extractor to automatically obtain features from input and XGBoost as a recognizer in the top level of the network to produce results. Experiments are implemented on the well-known MNIST and CIFAR-10 databases. The results prove that the new method performs better compared with other methods on the same databases, which verify the effectiveness of the proposed method in image classification problem.


Convolutional Neural Network eXtreme Gradient Boosting Image classification 



This research work is funded by the National Key Research and Development Project of China (2016YFB0801003), the Key Laboratory for Shanghai Integrated Information Security Management Technology Research.


  1. 1.
    Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, vol. 25(2), pp. 1097–1105 (2012)Google Scholar
  3. 3.
    Erhan, D., Szegedy, C., Toshev, A., et al.: Scalable object detection using deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2147–2154 (2014)Google Scholar
  4. 4.
    Diao, W., Sun, X., Zheng, X., et al.: Efficient saliency-based object detection in remote sensing images using deep belief networks. IEEE Geosci. Remote Sens. Lett. 13(2), 137–141 (2016)CrossRefGoogle Scholar
  5. 5.
    Li, C., Wei, W., Wang, J., Tang, W., Zhao, S.: Face recognition based on deep belief network combined with center-symmetric local binary pattern. In: Park, J., Jin, H., Jeong, Y.S., Khan, M. (eds.) Advanced Multimedia and Ubiquitous Engineering. LNEE, vol. 393, pp. 277–283. Springer, Singapore (2016). doi: 10.1007/978-981-10-1536-6_37 CrossRefGoogle Scholar
  6. 6.
    Taigman, Y., Yang, M., Ranzato, M.A., et al.: Deepface: closing the gap to human-level performance in face verification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1701–1708 (2014)Google Scholar
  7. 7.
    Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815–823 (2015)Google Scholar
  8. 8.
    Song, R., Chen, S., Deng, B., Li, L.: eXtreme gradient boosting for identifying individual users across different digital devices. In: Cui, B., Zhang, N., Xu, J., Lian, X., Liu, D. (eds.) WAIM 2016, Part I. LNCS, vol. 9658, pp. 43–54. Springer, Cham (2016). doi: 10.1007/978-3-319-39937-9_4 CrossRefGoogle Scholar
  9. 9.
    Mackey, L., Bryan, J., Mo, M.Y.: Weighted classification cascades for optimizing discovery significance in the HIGGSML challenge. In: NIPS 2014 Workshop on High-energy Physics and Machine Learning, pp. 129–134 (2015)Google Scholar
  10. 10.
    Bekkerman, R.: The present and the future of the KDD cup competition: an outsider’s perspectiveGoogle Scholar
  11. 11.
    Lecun, Y., Boser, B., Denker, J.S., et al.: Handwritten digit recognition with a back-propagation network. In: Advances in Neural Information Processing Systems, pp. 396–404 (1990)Google Scholar
  12. 12.
    Zipser, D., Andersen, R.A.: A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons. Nature 331(6158), 679–684 (1988)CrossRefGoogle Scholar
  13. 13.
    Chen, T., Guestrin, C.: Xgboost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794. ACM (2016)Google Scholar
  14. 14.
    Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  15. 15.
    Chan, T.H., Jia, K., Gao, S., et al.: Pcanet: a simple deep learning baseline for image classification. IEEE Trans. Image Process. 24(12), 5017–5032 (2015)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Ao, D.: Integration of Unsupervised Feature Learning and Neural Networks Applied to Image Recognition, pp. 19–37. South China University of Technology (2014)Google Scholar
  17. 17.
    Elleuch, M., Maalej, R., Kherallah, M.: A new design based-SVM of the CNN classifier architecture with dropout for offline arabic handwritten recognition. Procedia Comput. Sci. 80, 1712–1723 (2016)CrossRefGoogle Scholar
  18. 18.
    Le, Q., Sarlós, T., Smola, A.: Fastfood-approximating kernel expansions in loglinear time. In: Proceedings of the International Conference on Machine Learning (2013)Google Scholar
  19. 19.
    Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Krizhevsky, A.: Learning multiple layers of features from tiny images. Master’s thesis, pp. 30–35. University of Toronto (2009)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Xudie Ren
    • 1
  • Haonan Guo
    • 1
  • Shenghong Li
    • 1
    Email author
  • Shilin Wang
    • 1
  • Jianhua Li
    • 1
  1. 1.School of Cyber Space SecurityShanghai Jiao Tong UniversityShanghaiChina

Personalised recommendations