Online Multi-label Passive Aggressive Active Learning Algorithm Based on Binary Relevance

  • Xizhi Guo
  • Yongwei Zhang
  • Jianhua XuEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10638)


Online multi-label learning is an efficient classification paradigm in machine learning. However, traditional online multi-label methods often need requesting all class labels of each incoming sample, which is often human cost and time-consuming in labeling classification problem. In order to tackle these problems, in this paper, we present online multi-label passive aggressive active (MLPAA) learning algorithm by combining binary relevance (BR) decomposition strategy with online passive aggressive active (PAA) method. The proposed MLPAA algorithm not only uses the misclassified labels to update the classifier, but also exploits correctly classified examples with low prediction confidence. We perform extensive experimental comparison for our algorithm and the other methods using nine benchmark data sets. The encouraging results of our experiments validate the effectiveness of our proposed method.


Online active learning Multi-label classification Passive aggressive Binary relevance 



This work was supported by the Natural Science Foundation of China (NSFC) under Grant 61273246.


  1. 1.
    Hoi, S.C.H., Wang, J.L., Zhao, P.: LIBOL: a library for online learning algorithms. J. Mach. Learn. Res. 15, 495–499 (2014)zbMATHGoogle Scholar
  2. 2.
    Rosenblatt, F.: The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev. 65(1), 386–408 (1958)CrossRefGoogle Scholar
  3. 3.
    Crammer, K., Dekel, O., Keshet, J., et al.: Online passive-aggressive algorithms. J. Mach. Learn. Res. 7, 551–585 (2006)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Crammer, K., Kulesza, A., Dredze, M.: Adaptive regularization of weight vectors. Mach. Learn. 91(2), 155–187 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Crammer, K., Dredze, M., Pereira, F.: Exact convex confidence-weighted learning. In: Proceedings of the NIPS, pp. 345–352 (2008)Google Scholar
  6. 6.
    Liu, D., Zhang, P., Zheng, Q.: An efficient online active learning algorithm for binary classification. Pattern Recogn. Lett. 68(1), 22–26 (2015)CrossRefGoogle Scholar
  7. 7.
    Dasgupta, S., Kalai, A., Monteleoni, C.: Analysis of perceptron-based active learning. J. Mach. Learn. Res. 10, 281–299 (2009)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Lu, J., Zhao, P., Hoi, S.C.H.: Online passive aggressive active learning and its applications. In: Proceedings of the ACML, pp. 266–282 (2014)Google Scholar
  9. 9.
    Zhao, P., Hoi, S.C.H., Jin, R.: Double updating online learning. J. Mach. Learn. Res. 12, 1587–1615 (2011)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Crammer, K., Dredze, M., Kulesza, A.: Multi-class confidence weighted algorithms. In: Proceedings of the EMNLP, pp. 496–504 (2009)Google Scholar
  11. 11.
    Fink, M., Shwartz, S.S.; Singer, Y., et al.: Online multiclass learning by interclass hypothesis sharing. In: Proceedings of the ICML, pp. 313–320 (2006)Google Scholar
  12. 12.
    Crammer, K., Singer, Y.: A family of additive online algortihtms for category ranking. J. Mach. Learn. Res. 3, 1025–1058 (2003)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Higuchi, D., Ozawa, S.: A neural network model for online multi-task multi-label pattern recognition. In: Mladenov, V., Koprinkova-Hristova, P., Palm, G., Villa, A.E.P., Appollini, B., Kasabov, N. (eds.) ICANN 2013. LNCS, vol. 8131, pp. 162–169. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-40728-4_21 CrossRefGoogle Scholar
  14. 14.
    Park, S., Choi, S.: Online multi-label learning with accelerated nonsmooth stochastic gradient descent. In: ICASSP, pp. 3322–3326 (2013)Google Scholar
  15. 15.
    Zhang, X., Graepel, T., Herbrich, R.: Bayesian online learning for multi-label and multi-variate performance measures. In: Proceedings of the AISTATS, pp. 956–963 (2013)Google Scholar
  16. 16.
    Hua, X.S., et al.: Two-dimensional multi-label active learning with an efficient online adaptation model for image classification. IEEE Trans. Pattern Anal. Mach. Intell. 31(10), 1880–1897 (2009)CrossRefGoogle Scholar
  17. 17.
    Hua, K., Sheng, X., Qi, G.: Online multi-label active learning for large-scale multimedia annotation. Technical report from Microsoft Research (2008).
  18. 18.
    Gibaja, E., Ventura, S.: A tutorial on multi-label learning. ACM Comput. Surv. 47(3), 1–38 (2015). Article No. 51CrossRefGoogle Scholar
  19. 19.
    Herbrich, R., Minka, T., Graepel, T.: TrueskillTM: a Bayesian skill ranking system. In: Proceedings of the NIPS, pp. 569–576 (2006)Google Scholar
  20. 20.
    Cesa-Bianchi, N., Gentile, C., Zaniboni, L.: Worst-case analysis of selective sampling for linear classification. J. Mach. Learn. Res. 7, 1205–1230 (2006)MathSciNetzbMATHGoogle Scholar
  21. 21.
    Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge University Press, Cambridge (2006)CrossRefzbMATHGoogle Scholar
  22. 22.
    Zhang, M.L., Zhou, Z.H.: A review on multi-label learning algorithms. IEEE Trans. Knowl. Data Eng. 26(8), 1819–1837 (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.School of Computer Science and TechnologyNanjing Normal UniversityNanjingChina

Personalised recommendations