Learning from label proportions with pinball loss

  • Yong Shi
  • Limeng Cui
  • Zhensong Chen
  • Zhiquan QiEmail author
Original Article


Learning from label proportions is a new kind of learning problem which has drawn much attention in recent years. Different from the well-known supervised learning, it considers instances in bags and uses the label proportion of each bag instead of instance. As obtaining the instance label is not always feasible, it has been widely used in areas like modeling voting behaviors and spam filtering. However, learning from label proportions still suffers great challenges due to the inference of noise, the improper partition of bags and so on. In this paper, we propose a novel learning from label proportions method based on pinball loss, called “pSVM-pin”, to address the above issues. The pinball loss is introduced to generate an effective classifier in order to eliminate the impact of noise. Experimental results prove the precision of pSVM-pin compared with competing methods.


Learning from label proportions Label proportion Support vector machine Pinball loss 



We thank the anonymous reviewer for thoroughly reading our manuscript and providing helpful comments.This work is supported by National Natural Science Foundation of China (Grant nos. 91546201, 71331005, 71110107026, 61402429).


  1. 1.
    Hernández-González J, Inza I, Lozano JA (2015) A novel weakly supervised problem: learning from positive-unlabeled proportions. In: Puerta J et al (eds) Advances in artificial intelligence. Springer, Cham, pp 3–13CrossRefGoogle Scholar
  2. 2.
    Chapelle O, Schölkopf B, Zien A et al (2006) Semi-supervised learning. IEEE Transactions on Neural Networks 20(3):542–542CrossRefGoogle Scholar
  3. 3.
    Zhu X, Goldberg AB (2009) Introduction to semi-supervised learning. Synthesis lectures on artificial intelligence and machine learning 3(1):1–130CrossRefzbMATHGoogle Scholar
  4. 4.
    Andrews S, Tsochantaridis I, Hofmann T (2002) Support vector machines for multiple-instance learning. In: Advances in neural information processing systems, pp 561–568Google Scholar
  5. 5.
    Bunescu RC, Mooney RJ (2007) ​Multiple instance learning for sparse positive bags. In: Proceedings of the 24th international conference on machine learning. ACM, pp 105–112Google Scholar
  6. 6.
    Quadrianto N, Smola AJ, Caetano TS, Le QV (2009) ​Estimating labels from label proportions. J Mach Learn Res 10:2349–2374MathSciNetzbMATHGoogle Scholar
  7. 7.
    Rueping S (2010) SVM classifier estimation from group probabilities. In: Proceedings of the 27th international conference on machine learning (ICML-10), pp 911–918Google Scholar
  8. 8.
    Stolpe M, Morik K (2011) ​Learning from label proportions by optimizing cluster model selection. In: Gunopulos D, Hofmann T, Malerba D, Vazirgiannis M (eds) Machine learning and knowledge discovery in databases. Springer, Berlin, pp 349–364CrossRefGoogle Scholar
  9. 9.
    Yu F, Liu D, Kumar S, Tony J, Chang SF (2013) \(\propto\)SVM for learning with label proportions. In: Proceedings of the 30th international conference on machine learning, pp 504–512Google Scholar
  10. 10.
    Patrini G, Nock R, Caetano T, Rivera P (2014) (Almost) no label no cry. In: Advances in Neural Information Processing Systems, pp 190–198Google Scholar
  11. 11.
    Musicant DR, Christensen JM, Olson JF (2007) Supervised learning by training on aggregate outputs. Data mining, 2007. ICDM 2007. Seventh IEEE international conference on IEEE, pp 252–261Google Scholar
  12. 12.
    Chen T, Yu FX, Chen J, Cui Y, Chen YY, Chang SF (2014) Object-based visual sentiment concept analysis and application. In: Proceedings of the ACM international conference on multimedia. ACM, pp 367–376Google Scholar
  13. 13.
    Lai KT, Yu FX, Chen MS, Chang SF (2014) Video event detection by inferring temporal instance labels. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 2251–2258Google Scholar
  14. 14.
    Sweeney L (2002) k-anonymity: a model for protecting privacy. Int J Uncertain Fuzziness Knowl Based Syst 10(05):557MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Xiao X, Tao Y (2006) Anatomy: Simple and effective privacy preservation. In: Proceedings of the 32nd international conference on Very large data bases. VLDB Endowment, pp 139–150Google Scholar
  16. 16.
    Martin DJ, Kifer D, Machanavajjhala A, Gehrke J, Halpern JY (2007) Worst-case background knowledge for privacy-preserving data publishing. In: Data Engineering, 2007. ICDE 2007. IEEE 23rd International Conference on IEEE, pp 126–135Google Scholar
  17. 17.
    Kumari DA (2013) Slicing: a new approach to privacy preserving data publishing related to medical data-base using k-means clustering technique. Int J Adv Engg Res Technol 2(8)Google Scholar
  18. 18.
    Li XB, Sarkar S (2006) A tree-based data perturbation approach for privacy-preserving data mining. IEEE Trans Knowl Data Eng 18(9):1278CrossRefGoogle Scholar
  19. 19.
    Muralidhar K, Parsa R, Sarathy R (1999) A general additive data perturbation method for database security. Manag Sci 45(10):1399–1415CrossRefGoogle Scholar
  20. 20.
    Mitra P, Murthy C, Pal SK (2000) Data condensation in large databases by incremental learning with support vector machines. Pattern recognition, 2000. In: Proceedings of 15th international conference on, vol 2. IEEE, pp 708–711Google Scholar
  21. 21.
    Pan F, Zhang X, Wang W (2008) Crd: fast co-clustering on large datasets utilizing sampling-based matrix decomposition. In: Proceedings of the 2008 ACM SIGMOD international conference on Management of data. ACM, pp 173–184Google Scholar
  22. 22.
    Kück H, de Freitas N (2005) Learning about individuals from group statistics. In: Proceedings of the twenty-first conference on uncertainty in artificial intelligence. AUAI Press, Corvallis, pp 332–339Google Scholar
  23. 23.
    Hernández J, Inza I (2011) Learning naive Bayes models for multiple-instance learning with label proportions. In: Lozano JA, Gámez JA, Moreno JA (eds) Advances in Artificial Intelligence. Springer, Berlin, Heidelberg, pp 134–144CrossRefGoogle Scholar
  24. 24.
    Huang X, Shi L, Suykens JA (2015) Sequential minimal optimization for SVM with pinball loss. Neurocomputing 149:1596–1603CrossRefGoogle Scholar
  25. 25.
    Koenker R (2005) Quantile regression, vol 38. Cambridge University PressGoogle Scholar
  26. 26.
    Christmann A, Steinwart I (2007) How SVMs can estimate quantiles and the median. In: Advances in neural information processing systems, pp 305–312Google Scholar
  27. 27.
    Steinwart I, Christmann A et al (2011) Estimating conditional quantiles with the help of the pinball loss. Bernoulli 17(1):211–225MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Huang X, Shi L, Suykens J et al (2014) Support vector machine classifier with pinball loss. IEEE Trans Pattern Anal Mach Intell 36(5):984–997CrossRefGoogle Scholar
  29. 29.
    Huang X, Shi L, Suykens JA (2014) Solution path for PIN-SVM classifiers with positive and negative \(\tau\) values. IEEE transactions on neural networks and learning systemsGoogle Scholar
  30. 30.
    Tragante do OV, Fierens D, Blockeel H (2011) Instance-level accuracy versus bag-level accuracy in multi-instance learning. In: Proceedings of the 23rd Benelux conference on artificial intelligence (BNAIC), p 8Google Scholar
  31. 31.
    Moro S, Laureano R, Cortez P (2011) Using data mining for bank direct marketing: an application of the crisp-dm methodology. In: Proceedings of European Simulation and Modelling Conference-ESM'2011, pp 117–121Google Scholar
  32. 32.
    Yu FX, Choromanski K, Kumar S, Jebara T, Chang SF (2014) On Learning from Label Proportions. arXiv:1402.5902 (arXiv preprint)

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  1. 1.School of Computer and Control EngineeringUniversity of Chinese Academy of SciencesBeijingChina
  2. 2.School of Economics and ManagementUniversity of Chinese Academy of SciencesBeijingChina
  3. 3.Key Laboratory of Big Data Mining and Knowledge ManagementChinese Academy of SciencesBeijingChina
  4. 4.College of Information Science & TechnologyUniversity of Nebraska OmahaOmahaUSA
  5. 5.Research Center on Fictitious Economy & Data ScienceChinese Academy of SciencesBeijingChina

Personalised recommendations