Multi-label optimal margin distribution machine
- 40 Downloads
Multi-label support vector machine (Rank-SVM) is a classic and effective algorithm for multi-label classification. The pivotal idea is to maximize the minimum margin of label pairs, which is extended from SVM. However, recent studies disclosed that maximizing the minimum margin does not necessarily lead to better generalization performance, and instead, it is more crucial to optimize the margin distribution. Inspired by this idea, in this paper, we first introduce margin distribution to multi-label learning and propose multi-label Optimal margin Distribution Machine (mlODM), which optimizes the margin mean and variance of all label pairs efficiently. Extensive experiments in multiple multi-label evaluation metrics illustrate that mlODM outperforms SVM-style multi-label methods. Moreover, empirical study presents the best margin distribution and verifies the fast convergence of our method.
KeywordsOptimal margin distribution machine Multi-label learning Support vector machine Margin theory
This research was supported by the National Key R&D Program of China (2018YFB1004300), NSFC (61673201), and the Collaborative Innovation Center of Novel Software Technology and Industrialization.
- Elisseeff, A., & Weston, J. (2002). A kernel method for multi-labelled classification. In T. G. Dietterich, S. Becker and Z. Ghahramani (Eds.), Advances in neural information processing systems (pp. 681–687). MIT Press.Google Scholar
- Guo, Y., & Schuurmans, D. (2011). Adaptive large margin training for multilabel classification. In: W. Burgard and D. Roth (Eds.), 25th AAAI conference on artificial intelligence. San Francisco, CA: AAAI Press.Google Scholar
- Jiang, A., Wang, C., & Zhu, Y. (2008). Calibrated Rank-SVM for multi-label image categorization. In 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence) (pp. 1450–1455). IEEE.Google Scholar
- Lv, S. H., Wang, L., & Zhou, Z. H. (2018). Optimal margin distribution network. arXiv preprint arXiv:1812.10761
- McCallum, A. (1999). Multi-label text classification with a mixture model trained by EM. In AAAI workshop on text learning (pp. 1–7)Google Scholar
- Reyzin, L., & Schapire, R. E. (2006). How boosting the margin can also boost classifier complexity. In Proceedings of the 23rd international conference on machine learning (pp. 753–760). ACM.Google Scholar
- Schölkopf, B., & Smola, A. J. (2001). Learning with kernels: Support vector machines, regularization, optimization, and beyond. Cambridge: MIT Press.Google Scholar
- Sha, F., Saul, L. K., & Lee, D. D. (2002). Multiplicative updates for nonnegative quadratic programming in support vector machines. In S. Becker, S. Thrun and K. Obermayer (Eds.,) Advances in neural information processing systems (pp. 1041–1048). MIT Press.Google Scholar
- Tan, Z. H., Zhang, T., & Zhou, Z. H. (2019). Coreset stochastic variance-reduced gradient with application to optimal margin distribution machine. In 33rd AAAI conference on artificial intelligence.Google Scholar
- Wu, X. Z., & Zhou, Z. H. (2017). A unified view of multi-label performance measures. In Proceedings of the 34th international conference on machine learning (Vol. 70, pp. 3780–3788). JMLR.org.Google Scholar
- Zhang, T., & Zhou, Z. H. (2014b). Large margin distribution machine. In Proceedings of the 20th ACM SIGKDD international conference on knowledge discovery and data mining (pp. 313–322). ACM.Google Scholar
- Zhang, T., & Zhou, Z. H. (2017). Multi-class optimal margin distribution machine. In Proceedings of the 34th international conference on machine learning (Vol. 70, pp. 4063–4071). JMLR.org.Google Scholar
- Zhang, T., & Zhou, Z. H. (2018). Optimal margin distribution clustering. In 22nd AAAI conference on artificial intelligence.Google Scholar
- Zhang, T., & Zhou, Z. H. (2018a). Semi-supervised optimal margin distribution machines. In Jérôme Lang (ed.) Proceedings of the 27th international joint conference on artificial intelligence (pp. 3104–3110). Stockholm, Sweden: IJCAI.Google Scholar
- Zhou, Z. H. (2019). Abductive learning: Towards bridging machine learning and logical reasoning. Science China Information Sciences, 62(7), 76101.Google Scholar
- Zhang, T., & Zhou, Z. (2019). Optimal margin distribution machine. In IEEE Transactions on Knowledge and Data Engineering. https://doi.org/10.1109/TKDE.2019.2897662.