ML-SLSTSVM: a new structural least square twin support vector machine for multi-label learning

  • Meisam Azad-ManjiriEmail author
  • Ali Amiri
  • Alireza Saleh Sedghpour
Theoretical advances


Multi-label learning (MLL) is a special supervised learning task, where any single instance possibly belongs to several classes simultaneously. Nowadays, MLL methods are increasingly required by modern applications, such as protein function classification, speech recognition and textual data classification. In this paper, a structural least square twin support vector machine (SLSTSVM) classifier for multi-label learning is presented. This proposed ML-SLSTSVM focuses on the cluster-based structural information of the corresponding class in each optimization problem, which is vital for designing a good classifier in different real-world problems. This method is extended to a nonlinear version by the kernel trick. Experimental results demonstrate that proposed method is superior in generalization performance to other classifiers.


Multi-label learning Support vector machine Twin SVM Structural SVM Lest square SVM 



  1. 1.
    Sorower MS (2010) A literature survey on algorithms for multi-label learning, vol 18. Oregon State University, CorvallisGoogle Scholar
  2. 2.
    Wu J-S, Huang S-J, Zhou Z-H (2014) Genome-wide protein function prediction through multi-instance multi-label learning. IEEE/ACM Trans Comput Biol Bioinform 11(5):891–902Google Scholar
  3. 3.
    Wang X, Zhang W, Zhang Q, Li G-Z (2015) MultiP-SChlo: multi-label protein subchloroplast localization prediction with Chou’s pseudo amino acid composition and a novel multi-label classifier. Bioinformatics 31(16):2639–2645Google Scholar
  4. 4.
    Singh-Miller N, Collins M (2009) Learning label embeddings for nearest-neighbor multi-class classification with an application to speech recognition. In: Advances in neural information processing systems 22 (NIPS 2009), pp 1678–1686Google Scholar
  5. 5.
    Xu G, Lee H, Koo M-W, Seo J (2017) Convolutional neural network using a threshold predictor for multi-label speech act classification. In: 2017 IEEE international conference on big data and smart computing (BigComp), 2017, pp 126–130Google Scholar
  6. 6.
    Wu B, Zhong E, Horner A, Yang Q (2014) Music emotion recognition by multi-label multi-layer multi-instance multi-view learning. In: Proceedings of the 22nd ACM international conference on multimedia, 2014, pp 117–126Google Scholar
  7. 7.
    Trohidis K, Tsoumakas G, Kalliris G, Vlahavas IP (2008) Multi-label classification of music into emotions. ISMIR 8:325–330Google Scholar
  8. 8.
    Liu Y, Wen K, Gao Q, Gao X, Nie F (2018) SVM based multi-label learning with missing labels for image annotation, vol 78. Elsevier Ltd., AmsterdamGoogle Scholar
  9. 9.
    Wu J, Yu Y, Huang C, Yu K (2015) Deep multiple instance learning for image classification and auto-annotation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp 3460–3469Google Scholar
  10. 10.
    Tsoumakas G, Katakis I (2007) Multi-label classification: an overview. Int J Data Warehous Min 3(3):1–13Google Scholar
  11. 11.
    Zhang M-L, Zhou Z-H (2014) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837Google Scholar
  12. 12.
    Fürnkranz J, Hüllermeier E, Loza Mencía E, Brinker K (2008) Multilabel classification via calibrated label ranking. Mach Learn 73(2):133–153Google Scholar
  13. 13.
    Elisseeff A, Weston J (2001) A kernel method for multi-labelled classification. In: Advances in neural information processing systems 14 (NIPS 2001), pp 681–687Google Scholar
  14. 14.
    Xu J (2013) Fast multi-label core vector machine. Pattern Recognit 46(3):885–898zbMATHGoogle Scholar
  15. 15.
    Chen WJ, Shao YH, Li CN, Deng NY (2016) MLTSVM: a novel twin support vector machine to multi-label learning. Pattern Recognit 52:61–74zbMATHGoogle Scholar
  16. 16.
    Zhang ML, Zhou ZH (2007) ML-KNN: a lazy learning approach to multi-label learning. Pattern Recognit 40(7):2038–2048zbMATHGoogle Scholar
  17. 17.
    Zhang M-L (2009) Ml-rbf: RBF neural networks for multi-label learning. Neural Process Lett 29(2):61–74Google Scholar
  18. 18.
    Zhang M-L, Zhou Z-H (2006) Multilabel neural networks with applications to functional genomics and text categorization. IEEE Trans Knowl Data Eng 18(10):1338–1351Google Scholar
  19. 19.
    Clare A, King RD (2001) Knowledge discovery in multi-label phenotype data. In: European conference on principles of data mining and knowledge discovery, 2001, pp 42–53Google Scholar
  20. 20.
    Vens C, Struyf J, Schietgat L, Džeroski S, Blockeel H (2008) Decision trees for hierarchical multi-label classification. Mach Learn 73(2):185–214Google Scholar
  21. 21.
    Qi Z, Tian Y, Shi Y (2013) Structural twin support vector machine for classification. Knowl Based Syst 43:74–81Google Scholar
  22. 22.
    Suykens JAK, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300Google Scholar
  23. 23.
    Arun Kumar M, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543Google Scholar
  24. 24.
    Hüllermeier E, Fürnkranz J, Cheng W, Brinker K (2008) Label ranking by learning pairwise preferences. Artif Intell 172(16–17):1897–1916MathSciNetzbMATHGoogle Scholar
  25. 25.
    Sun X et al (2016) ELM-ML: study on multi-label classification using extreme learning machine. In: Proceedings of ELM-2015, vol 2. Springer, 2016, pp 107–116Google Scholar
  26. 26.
    Schapire RE, Singer Y (2000) BoosTexter: a boosting-based system for text categorization. Mach Learn 39(2–3):135–168zbMATHGoogle Scholar
  27. 27.
    Wang Y et al (2017) A multi-label learning method for efficient affective detection. In: 2017 IEEE EMBS international conference on biomedical and health informatics, pp 61–64Google Scholar
  28. 28.
    Reyes O, Morell C, Ventura S (2018) Effective active learning strategy for multi-label learning. Neurocomputing 273:494–508Google Scholar
  29. 29.
    Li Z, Tang J, Mei T (2018) Deep collaborative embedding for social image understanding. IEEE Trans Pattern Anal Mach Intell. Google Scholar
  30. 30.
    Zhu J, Liao S, Lei Z, Li SZ (2017) Multi-label convolutional neural network based pedestrian attribute classification. Image Vis Comput 58:224–229Google Scholar
  31. 31.
    Zhuang N, Yan Y, Chen S, Wang H, Shen C (2018) Multi-label learning based deep transfer neural network for facial attribute classification. Pattern Recognit 80:225–240Google Scholar
  32. 32.
    Li Z, Tang J (2017) Weakly supervised deep matrix factorization for social image understanding. IEEE Trans Image Process 26(1):276–288MathSciNetGoogle Scholar
  33. 33.
    Cakir E, Heittola T, Huttunen H, Virtanen T (2015) Multi-label vs. combined single-label sound event detection with deep neural networks. In: 2015 23rd European signal processing conference (EUSIPCO), 2015, pp 2551–2555Google Scholar
  34. 34.
    Barutcuoglu Z, Schapire RE, Troyanskaya OG (2006) Hierarchical multi-label prediction of gene function. Bioinformatics 22(7):830–836Google Scholar
  35. 35.
    Zhang W, Yan J, Wang X, Zha H (2018) Deep extreme multi-label learning. In: Proceedings of the 2018 ACM on international conference on multimedia retrieval. ACM, pp 100–107Google Scholar
  36. 36.
    Prabhu Y, Varma M (2014) Fastxml: a fast, accurate and stable tree-classifier for extreme multi-label learning. In: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, 2014, pp 263–272Google Scholar
  37. 37.
    Weston J, Makadia A, Yee H (2013) Label partitioning for sublinear ranking. In: International conference on machine learning, 2013, pp 181–189Google Scholar
  38. 38.
    Xu C, Tao D, Xu C (2016) Robust extreme multi-label learning. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 2016, pp 1275–1284Google Scholar
  39. 39.
    Bhatia K, Jain H, Kar P, Varma M, Jain P (2015) Sparse local embeddings for extreme multi-label classification. In: Advances in neural information processing systems 28 (NIPS 2015), pp 730–738Google Scholar
  40. 40.
    Jayadeva, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910zbMATHGoogle Scholar
  41. 41.
    Vapnik VN (1999) An overview of statistical learning theory. IEEE Trans Neural Netw 10(5):988–999Google Scholar
  42. 42.
    Hofmann T, Schölkopf B, Smola AJ (2008) Kernel methods in machine learning. Ann Stat 36(3):1171–1220MathSciNetzbMATHGoogle Scholar
  43. 43.
    Auria L, Moro R (2008) Support vector machines (SVM) as a technique for solvency analysis. DIW Berlin discussion paper no. 811. Available at SSRN: or
  44. 44.
    Xue H, Chen S, Yang Q (2011) Structural regularized support vector machine: a framework for structural large margin classifier. IEEE Trans Neural Netw 22(4):573–587Google Scholar
  45. 45.
    Shao Y-H, Zhang C-H, Wang X-B, Deng N-Y (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22(6):962–968Google Scholar
  46. 46.
    Fung G, Mangasarian OL (2001) Proximal support vector machine classifiers. In: Proceeding of ACM SIGKDD international conference on knowledge discovery and data mining—KDD’01, pp 77–86Google Scholar
  47. 47.
    Tsoumakas G, Spyromitros-Xioufis E, Vilcek J, Vlahavas I (2011) Mulan: a java library for multi-label learning. J Mach Learn Res 12(Jul):2411–2414MathSciNetzbMATHGoogle Scholar
  48. 48.
    Nasiri JA, Charkari NM, Jalili S (2015) Least squares twin multi-class classification support vector machine. Pattern Recognit 48(3):984–992zbMATHGoogle Scholar
  49. 49.
    Wilcoxon F (1945) Individual comparisons by ranking methods. Biometrics Bull 1(6):80–83Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  • Meisam Azad-Manjiri
    • 1
    Email author
  • Ali Amiri
    • 1
  • Alireza Saleh Sedghpour
    • 2
  1. 1.Department of Computer EngineeringUniversity of ZanjanZanjanIran
  2. 2.Department of Computer EngineeringIran University of Science and TechnologyTehranIran

Personalised recommendations