An Advanced Least Squares Twin Multi-class Classification Support Vector Machine for Few-Shot Classification

  • Yu Li
  • Zhonggeng Liu
  • Huadong PanEmail author
  • Jun Yin
  • Xingming Zhang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11936)


In classification tasks, deep learning methods yield high performance. However, owing to lack of enough annotated data, deep learning methods often underperformed. Therefore, we propose an advance version of least squares twin multi-class classification support vector machine (ALST-KSVC) which leads to low computational complexity and comparable accuracy based on LST-KSVC for few-shot classification. In ALST-KSVC, we modified optimization problems to construct a new “1-versus-1-versus-1” structure, proposed a new decision function, and constructed smaller number of classifiers than our baseline LST-KSVC. We empirically demonstrate that the proposed method has better classification accuracy than LST-KSVC. Especially, ALST-KSVC achieves the state-of-the-art performance on MNIST, USPS, Amazon, Caltech image datasets and Iris, Teaching evaluation, Balance, Wine, Transfusion UCI datasets.


Few-shot Multi-classes classification Support vector machine 


  1. 1.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations, May 2015Google Scholar
  2. 2.
    Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on International Conference on Machine Learning. (2015)Google Scholar
  3. 3.
    He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition (2015)Google Scholar
  4. 4.
    Huang, G., Liu, Z., Laurens, V.D.M., et al.: Densely connected convolutional networks (2016)Google Scholar
  5. 5.
    Kumar, M.A., Gopal, M.: Least squares twin support vector machines for pattern classification. Expert Syst. Appl. 36(4), 7535–7543 (2009)CrossRefGoogle Scholar
  6. 6.
    Shao, Y.H., Deng, N.Y., Yang, Z.M.: Least squares recursive projection twin support vector machine for classification. Int. J. Mach. Learn. Cybern. 45(6), 2299–2307 (2012)zbMATHGoogle Scholar
  7. 7.
    Lecun, Y., Bottou, L., Bengio, Y., et al.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  8. 8.
    Wu, Y.C., Lee, Y.S., Yang, J.C.: Robust and efficient multiclass SVM models for phrase pattern recognition. Pattern Recogn. 41(9), 2874–2889 (2008)CrossRefGoogle Scholar
  9. 9.
    Liu, R., Wang, Y., Baba, T., et al.: SVM-based active feedback in image retrieval using clustering and unlabeled data. Pattern Recogn. 41(8), 2645–2655 (2008)CrossRefGoogle Scholar
  10. 10.
    Jayadeva, Khemchandani, R., Chandra, S.: Twin support vector machines for pattern classification. IEEE Trans. Pattern Anal. Mach. Intell. 29(5), 905–910 (2007)CrossRefGoogle Scholar
  11. 11.
    Shao, Y.H., Zhang, C.H., Wang, X.B., et al.: Improvements on twin support vector machines. IEEE Trans. Neural Netw. 22(6), 962–968 (2011)CrossRefGoogle Scholar
  12. 12.
    Bottou, L., Cortes, C., Denker, J.S., et al.: Comparison of classifier methods: a case study in handwritten digit recognition. In: International Conference on Pattern Recognition (1994)Google Scholar
  13. 13.
    Krebel, U.: Pairwise classification and support vector machines. In: Advances in Kernel Methods. MIT Press (1999)Google Scholar
  14. 14.
    Angulo, C., Parra, X., Català, A.: K-SVCR. A support vector machine for multi-class classification. Neurocomputing 55(1–2), 57–77 (2003)CrossRefGoogle Scholar
  15. 15.
    Xu, Y., Guo, R., Wang, L.: A twin multi-class classification support vector machine. Cogn. Comput. 5(4), 580–588 (2013)CrossRefGoogle Scholar
  16. 16.
    Nasiri, J.A., Charkari, N.M., Jalili, S.: Least squares twin multi-class classification support vector machine. Pattern Recogn. 48(3), 984–992 (2015)CrossRefGoogle Scholar
  17. 17.
    Cristianini, N.: An introduction to support vector machines and other kernel-based learning methods. Kybernetes 32(1), 1–28 (2001)zbMATHGoogle Scholar
  18. 18.
    Moore, B.C.: Principal component analysis in linear systems: controllability, observability, and model reduction. IEEE Trans. Autom. Control 26(1), 17–32 (2003)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Saenko, K., Kulis, B., Fritz, M., Darrell, T.: Adapting visual category models to new domains. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 213–226. Springer, Heidelberg (2010). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Yu Li
    • 1
  • Zhonggeng Liu
    • 1
  • Huadong Pan
    • 1
    Email author
  • Jun Yin
    • 1
  • Xingming Zhang
    • 1
  1. 1.Advanced Research Institute of Zhejiang Dahua Technology Co. Ltd.HangzhouChina

Personalised recommendations