Centering SVDD for Unsupervised Feature Representation in Object Classification

  • Dong Wang
  • Xiaoyang Tan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8228)


Learning good feature representation from unlabeled data has attracted researchers great attention recently. Among others, K-means clustering algorithm is popularly used to map the input data into a feature representation, by finding the nearest centroid for each input point. However, this ignores the density information of each cluster completely and the resulting representation may be too terse. In this paper, we proposed a SVDD (Support Vector Data Description) based method to address these issues. The key idea of our method is to use SVDD to measure the density of each cluster resulted from K-Means clustering, based on which a robust feature representation can be derived. For this purpose, we add a new constraint to the original SVDD objective function to make the model align better with the data. In addition, we show that our modified SVDD can be solved very efficiently as a linear programming problem, instead of as a quadratic one. The effectiveness and feasibility of the proposed method is verified on two object classification databases with promising results.


Feature learning K-means Support Vector Data Description(SVDD) C-SVDD object classification 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bengio, Y., Courville, A., Vincent, P.: Representation learning: A review and new perspectives. arXiv preprint arXiv:12065538 (2012)Google Scholar
  2. 2.
    Le, Q.V., Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G.S., et al.: Building high-level features using large scale unsupervised learning. arXiv preprint arXiv:11126209 (2011)Google Scholar
  3. 3.
    Agarwal, A., Triggs, B.: Hyperfeatures – multilevel local coding for visual recognition. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part I. LNCS, vol. 3951, pp. 30–43. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  4. 4.
    Jarrett, K., Kavukcuoglu, K., Ranzato, M., LeCun, Y.: What is the best multi-stage architecture for object recognition? In: 2009 IEEE 12th International Conference on Computer Vision, pp. 2146–2153. IEEE (2009)Google Scholar
  5. 5.
    Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Cueto, M.A., Morton, J., Sturmfels, B.: Geometry of the restricted Boltzmann machine. In: Viana, M., Wynn, H. (eds.) Algebraic Methods in Statistics and Probability. AMS, Contemporary Mathematics, vol. 516, pp. 135–153 (2010)Google Scholar
  7. 7.
    Coates, A., Lee, H., Ng, A.Y.: An analysis of single-layer networks in unsupervised feature learning. Ann Arbor 1001, 48109 (2010)Google Scholar
  8. 8.
    Tax, D.M., Duin, R.P.: Support vector data description. Machine Learning 54(1), 45–66 (2004)CrossRefzbMATHGoogle Scholar
  9. 9.
    Xu, J., Yao, J., Ni, L.: Fault detection based on SVDD and cluster algorithm. In: 2011 International Conference on Electronics, Communications and Control (ICECC), pp. 2050–2052. IEEE (2011)Google Scholar
  10. 10.
    Niu, X.X., Suen, C.Y.: A novel hybrid CNN–SVM classifier for recognizing handwritten digits. Pattern Recognition 45(4), 1318–1325 (2012)CrossRefGoogle Scholar
  11. 11.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  12. 12.
    Martinez, A.M.: The AR face database. CVC Technical Report 24 (1998)Google Scholar
  13. 13.
    Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images. Master’s thesis, Department of Computer Science, University of Toronto (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Dong Wang
    • 1
  • Xiaoyang Tan
    • 1
  1. 1.Department of Computer Science and TechnologyNanjing University of Aeronautics and AstronauticsNanjingP.R. China

Personalised recommendations