Deep-PUMR: Deep Positive and Unlabeled Learning with Manifold Regularization

  • Xingyu Chen
  • Fanghui Liu
  • Enmei Tu
  • Longbing Cao
  • Jie YangEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11301)


Training a binary classifier only on positive and unlabeled examples (i.e., the PU learning) is an important yet challenging issue, widely seen in many problems in which it is difficult to obtain negative examples. Existing methods for handling this challenge often perform unsatisfactorily, since they often ignore the relations between positive and unlabeled examples and are also limited to the traditional shallow learning frameworks. Therefore, this work proposes a new approach: Deep Positive and Unlabeled learning with Manifold Regularization (Deep-PUMR), which integrates the manifold regularization with deep neural networks to address the above issues with classic PU learning. Deep-PUMR holds two major advantages: (i) Our method exploits the manifold properties of data distribution to capture the relationship of positive and unlabeled examples; (ii) The adopted deep network enables Deep-PUMR with strong learning ability, especially on large-scale datasets. Extensive experiments on five diverse datasets demonstrate that Deep-PUMR achieves the state-of-the-art performance in comparison with classic PU learning algorithms and risk estimators.


PU learning Deep neural network Manifold learning 



This research is partly supported by NSFC, China (No: 61572315, 6151101179) and 973 Plan, China (No. 2015CB856004).


  1. 1.
    Belkin, M., Niyogi, P., Sindhwani, V.: Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. JMLR 7(1), 2399–2434 (2006)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Belkin, M., Niyogi, P., Sindhwani, V.: On manifold regularization. In: AISTATS, p. 1 (2005)Google Scholar
  3. 3.
    Du Plessis, M., Niu, G., Sugiyama, M.: Convex formulation for learning from positive and unlabeled data. In: ICML, pp. 1386–1394 (2015)Google Scholar
  4. 4.
    Du Plessis, M.C., Niu, G., Sugiyama, M.: Analysis of learning from positive and unlabeled data. In: NIPS, pp. 703–711 (2014)Google Scholar
  5. 5.
    Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. JMLR 12(Jul), 2121–2159 (2011)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Elhamifar, E., Vidal, R.: Sparse manifold clustering and embedding. In: NIPS, pp. 55–63 (2011)Google Scholar
  7. 7.
    Elkan, C., Noto, K.: Learning classifiers from only positive and unlabeled data. In: KDD, pp. 213–220 (2008)Google Scholar
  8. 8.
    Gong, C., Liu, T., Tao, D., Fu, K., Tu, E., Yang, J.: Deformed graph laplacian for semisupervised learning. TNNLS 26(10), 2261–2274 (2015)MathSciNetGoogle Scholar
  9. 9.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)Google Scholar
  10. 10.
    Hsieh, C.J., Natarajan, N., Dhillon, I.: PU learning for matrix completion. In: JMLR, pp. 2445–2453 (2015)Google Scholar
  11. 11.
    Jaafar, H.F., Nandi, A.K., Al-Nuaimy, W.: Automated detection and grading of hard exudates from retinal fundus images. In: EUSIPCO, pp. 66–70. IEEE (2011)Google Scholar
  12. 12.
    Kiryo, R., Niu, G., du Plessis, M.C., Sugiyama, M.: Positive-unlabeled learning with non-negative risk estimator. NIPS (2017)Google Scholar
  13. 13.
    Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images (2009)Google Scholar
  14. 14.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  15. 15.
    Lee, W.S., Liu, B.: Learning with positive and unlabeled examples using weighted logistic regression. In: ICML, pp. 448–455 (2003)Google Scholar
  16. 16.
    Li, X.L., Yu, P.S., Liu, B., Ng, S.K.: Positive unlabeled learning for data stream classification. In: SDM, pp. 259–270 (2009)Google Scholar
  17. 17.
    Li, X., Liu, B.: Learning to classify texts using positive and unlabeled data. In: IJCAI, pp. 587–592 (2003)Google Scholar
  18. 18.
    Liu, B., Lee, W.S., Yu, P.S., Li, X.: Partially supervised classification of text documents. In: ICML, pp. 387–394 (2002)Google Scholar
  19. 19.
    Menon, A., Van Rooyen, B., Ong, C.S., Williamson, B.: Learning from corrupted binary labels via class-probability estimation. In: ICML, pp. 125–134 (2015)Google Scholar
  20. 20.
    Nadler, B., Lafon, S., Kevrekidis, I., Coifman, R.R.: Diffusion maps, spectral clustering and eigenfunctions of fokker-planck operators. In: NIPS, pp. 955–962 (2006)Google Scholar
  21. 21.
    Niu, G., du Plessis, M.C., Sakai, T., Ma, Y., Sugiyama, M.: Theoretical comparisons of positive-unlabeled learning against positive-negative learning. In: NIPS, pp. 1199–1207 (2016)Google Scholar
  22. 22.
    Redmon, J., Farhadi, A.: Yolo9000: better, faster, stronger. In: CVPR (2016)Google Scholar
  23. 23.
    Rothe, R., Timofte, R., Gool, L.V.: Deep expectation of real and apparent age from a single image without facial landmarks. IJCV 126, 144–157 (2016)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Ruder, S.: An overview of gradient descent optimization algorithms. arXiv (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Xingyu Chen
    • 1
  • Fanghui Liu
    • 1
  • Enmei Tu
    • 1
  • Longbing Cao
    • 2
  • Jie Yang
    • 1
    Email author
  1. 1.Institute of Image Processing and Pattern RecognitionShanghai Jiao Tong UniversityShanghaiChina
  2. 2.Advanced Analytics InstituteUniversity of Technology at SydneyUltimoAustralia

Personalised recommendations