Skip to main content
SpringerLink
Log in
Book cover

Robust Representation for Data Analytics pp 17–44Cite as

Robust Graph Construction

  • Chapter
  • First Online:

Part of the book series: Advanced Information and Knowledge Processing ((AI&KP))

Abstract

Graphs have been widely applied in modeling the relationships and structures in real-world applications. Graph construction is the most critical part in these models, while how to construct an effective graph is still an open problem. In this chapter, we propose a novel approach to graph construction based on two observations. First, by virtue of recent advances in low-rank subspace recovery, the similarity between every two samples evaluated in the low-rank code space is more robust than that in the sample space. Second, a sparse and balanced graph can greatly increase the performance of learning tasks, such as label propagation in graph based semi-supervised learning. The k-NN sparsification can provide fast solutions to constructing unbalanced sparse graphs, and b-matching constraint is a necessary route for generating balanced graphs. These observations motivate us to jointly learn the low-rank codes and balanced (or unbalanced) graph simultaneously. In particular, two non-convex models are built by incorporating k-NN constraint and b-matching constraint into the low-rank representation model, respectively. We design a majorization-minimization augmented Lagrange multiplier (MM-ALM) algorithm to solve the proposed models. Extensive experimental results on four image databases demonstrate the superiority of our graphs over several state-of-the-art graphs in data clustering, transductive and inductive semi-supervised learning.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   129.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Semi-supervised learning can be either transductive or inductive. Transductive model only works on the labeled and unlabeled training samples, and it cannot deal with unseen data. Inductive model can naturally handle unseen data [55, 57].

References

  1. Anand, R., Reddy, C.K.: Graph-based clustering with constraints. In: The Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD), pp. 51–62 (2011)

    Google Scholar 

  2. Bach, F.: Consistency of trace norm minimization. J. Mach. Learn. Res. 9, 1019–1048 (2008)

    MathSciNet  MATH  Google Scholar 

  3. Bellhumeur, P.N., Hespanha, J.P., Kriegeman, D.J.: Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 711–720 (1997)

    Google Scholar 

  4. Cai, D., He, X., Han, J.: Semi-supervised discriminant analysis. In: International Conference on Computer Vision (ICCV), pp. 1–7 (2007)

    Google Scholar 

  5. Candès, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM 58(3), 11 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  6. Chapelle, O., Schölkopf, B., Zien, A.: Semi-supervised Learning. MIT Press, Cambridge (2006)

    Book  Google Scholar 

  7. Chen, K., Wang, S.: Regularized boost for semi-supervised learning. In: Advances in Neural Information Processing Systems (NIPS) (2007)

    Google Scholar 

  8. Cheng, H., Liu, Z., Yang, J.: Sparsity induced similarity measure for label propagation. In: International Conference on Computer Vision (ICCV), pp. 317–324 (2009)

    Google Scholar 

  9. Culp, M., Michailidis, G.: Graph-based semisupervised learning. IEEE Trans. Pattern Anal. Mach. Intell. 30(1), 174–179 (2008)

    Article  Google Scholar 

  10. de Sousa, C.A.R., Rezende, S.O., Batista, G.E.A.P.A.: Influence of graph construction on semi-supervised learning. In: The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML/PKDD) (3), pp. 160–175 (2013)

    Google Scholar 

  11. Georghiades, A., Belhumeur, P., Kriegman, D.: From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 643–660 (2001)

    Article  Google Scholar 

  12. Goldberg, A.B., Zhu, X., Wright, S.J.: Dissimilarity in graph-based semi-supervised classification. J. Mach. Learn. Res. Proc. Track 2, 155–162 (2007)

    Google Scholar 

  13. He, J., Carbonell, J.G., Liu, Y.: Graph-based semi-supervised learning as a generative model. In: International Joint Conference on Artificial Intelligence (IJCAI), pp. 2492–2497 (2007)

    Google Scholar 

  14. He, R., Zheng, W., Hu, B., Kong, X.: Nonnegative sparse coding for discriminative semi-supervised learning. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2849–2856 (2011)

    Google Scholar 

  15. Huang, B., Jebara, T.: Fast b-matching via sufficient selection belief propagation. J. Mach. Learn. Res. Proc. Track pp. 361–369 (2011)

    Google Scholar 

  16. Hull, J.: A database for handwritten text recognition research. IEEE Trans. Pattern Anal. Mach. Intell. 16(5), 550–554 (1994)

    Article  Google Scholar 

  17. Iwata, T., Duh, K.: Bidirectional semi-supervised learning with graphs. In: The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD) (2), pp. 293–306 (2012)

    Google Scholar 

  18. Jebara, T., Wang, J., Chang, S.-F.: Graph construction and b-matching for semi-supervised learning. In: International Conference on Machine Learning (ICML), pp. 441–448 (2009)

    Google Scholar 

  19. Kveton, B., Valko, M., Rahimi, A., Huang, L.: Semi-supervised learning with max-margin graph cuts. J. Mach. Learn. Res. Proc. Track 9, 421–428 (2010)

    Google Scholar 

  20. Li, L., Li, S., Fu, Y.: Discriminative dictionary learning with low-rank regularization for face recognition. In: IEEE International Conference on Automatic Face and Gesture Recognition (FG), pp. 1–6 (2013)

    Google Scholar 

  21. Li, S., Fu, Y.: Low-rank coding with b-matching constraint for semi-supervised classification. In: International Joint Conference on Artificial Intelligence (IJCAI), pp. 1472–1478 (2013)

    Google Scholar 

  22. Li, S., Fu, Y.: Robust subspace discovery through supervised low-rank constraints. In: SIAM International Conference on Data Mining (SDM), pp. 163–171 (2014)

    Google Scholar 

  23. Li, S., Fu, Y.: Learning balanced and unbalanced graphs via low-rank coding. IEEE Trans. Knowl. Data Eng. 27(5), 1274–1287 (2015)

    Article  Google Scholar 

  24. Li, S., Fu, Y.: Learning robust and discriminative subspace with low-rank constraints. IEEE Trans. Neural Netw. Learn. Syst. 27(11), 2160–2173 (2016)

    Article  MathSciNet  Google Scholar 

  25. Li, S., Fu, Y.: Unsupervised transfer learning via low-rank coding for image clustering. In: International Joint Conference on Neural Networks, pp. 1795–1802. IEEE (2016)

    Google Scholar 

  26. Li, S., Li, K., Fu, Y.: Self-taught low-rank coding for visual learning. IEEE Trans. Neural Netw. Learn. Syst. (2017)

    Google Scholar 

  27. Li, S., Shao, M., Fu, Y.: Locality linear fitting one-class SVM with low-rank constraints for outlier detection. In: International Joint Conference on Neural Networks (IJCNN), pp. 676–683 (2014)

    Google Scholar 

  28. Li, S., Shao, M., Fu, Y.: Multi-view low-rank analysis for outlier detection. In: Proceedings of the SIAM International Conference on Data Mining, pp. 748–756. SIAM (2015)

    Google Scholar 

  29. Lin, Z., Chen, M., Ma, Y.: The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrix. In: Technical Report, UIUC (2009)

    Google Scholar 

  30. Lin, Z., Liu, R., Su, Z.: Linearized alternating direction method with adaptive penalty for low-rank representation. In: Advances in Neural Information Processing Systems (NIPS), pp. 612–620 (2011)

    Google Scholar 

  31. Liu, G., Lin, Z., Yan, S., Sun, J., Yu, Y., Ma, Y.: Robust recovery of subspace structures by low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 171–184 (2013)

    Article  Google Scholar 

  32. Liu, G., Lin, Z., Yu, Y.: Robust subspace segmentation by low-rank representation. In: International Conference on Machine Learning (ICML), pp. 663–670 (2010)

    Google Scholar 

  33. Liu, J., Wang, C., Danilevsky, M., Han, J.: Large-scale spectral clustering on graphs. In: International Joint Conference on Artificial Intelligence (IJCAI), pp. 1486–1492 (2013)

    Google Scholar 

  34. Liu, W., Wang, J., Chang, S.-F.: Robust and scalable graph-based semisupervised learning. Proc. IEEE 100(9), 2624–2638 (2012)

    Article  Google Scholar 

  35. Ni, B., Yan, S., Kassim, A.A.: Learning a propagable graph for semisupervised learning: classification and regression. IEEE Trans. Knowl. Data Eng. 24(1), 114–126 (2012)

    Article  Google Scholar 

  36. Nie, F., Wang, H., Huang, H., Ding, C.H.Q.: Unsupervised and semi-supervised learning via l1-norm graph. In: International Conference on Computer Vision (ICCV), pp. 2268–2273 (2011)

    Google Scholar 

  37. Samaria, F., Harter, A.: Parameterisation of a stochastic model for human face identification. In: Proceedings of the Second IEEE Workshop on Applications of Computer Vision, pp. 138–142 (1994)

    Google Scholar 

  38. Shang, F., Jiao, L., Liu, Y., Tong, H.: Semi-supervised learning with nuclear norm regularization. Pattern Recogn. 46(8), 2323–2336 (2013)

    Article  MATH  Google Scholar 

  39. Shang, F., Jiao, L., Liu, Y., Wang, F.: Learning spectral embedding via iterative eigenvalue thresholding. In: ACM International Conference on Information and Knowledge Management (CIKM), pp. 1507–1511 (2012)

    Google Scholar 

  40. Shang, F., Jiao, L., Wang, F.: Semi-supervised learning with mixed knowledge information. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pp. 732–740 (2012)

    Google Scholar 

  41. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000)

    Article  Google Scholar 

  42. Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression database. IEEE Trans. Pattern Anal. Mach. Intell. 25(12), 1615–1618 (2003)

    Article  Google Scholar 

  43. Sokol, M., Avrachenkov, K., Gonçalves, P., Mishenin, A.: Generalized optimization framework for graph-based semi-supervised learning. In: SIAM International Conference on Data Mining (SDM), pp. 966–974 (2012)

    Google Scholar 

  44. Talwalkar, A., Mackey, L.W., Mu, Y., Chang, S.-F., Jordan, M.I.: Distributed low-rank subspace segmentation. In: International Conference on Computer Vision (ICCV), pp. 3543–3550 (2013)

    Google Scholar 

  45. Wang, F., Li, P., König, A.C., Wan, M.: Improving clustering by learning a bi-stochastic data similarity matrix. Knowl. Inf. Syst. 32(2), 351–382 (2012)

    Article  Google Scholar 

  46. Wang, F., Zhang, C.: Label propagation through linear neighborhoods. IEEE Trans. Knowl. Data Eng. 20(1), 55–67 (2008)

    Article  Google Scholar 

  47. Wang, J., Xia, Y.: Fast graph construction using auction algorithm. In: The Conference on Uncertainty in Artificial Intelligence (UAI), pp. 873–882 (2012)

    Google Scholar 

  48. Wang, S., Liu, D., Zhang, Z.: Nonconvex relaxation approaches to robust matrix recovery. In: International Joint Conference on Artificial Intelligence (IJCAI), pp. 1764–1770 (2013)

    Google Scholar 

  49. Wright, J., Yang, A.Y., Ganesh, A., Sastry, S., Ma, Y.: Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 31(2), 210–227 (2009)

    Article  Google Scholar 

  50. Yan, S., Wang, H.: Semi-supervised learning by sparse representation. In: SIAM International Conference on Data Mining (SDM), pp. 792–801 (2009)

    Google Scholar 

  51. Zhang, K., Wang, Q., Lan, L., Sun, Y., Marsic, I.: Sparse semi-supervised learning on low-rank kernel. Neurocomputing 129:265–272 (2014)

    Article  Google Scholar 

  52. Zhang, T., Ji, R., Liu, W., Tao, D., Hua, G.: Semi-supervised learning with manifold fitted graphs. In: International Joint Conference on Artificial Intelligence (IJCAI), pp. 1896–1902 (2013)

    Google Scholar 

  53. Zhang, Z., Chow, T.W.S., Zhao, M.-B.: Trace ratio optimization-based semi-supervised nonlinear dimensionality reduction for marginal manifold visualization. IEEE Trans. Knowl. Data Eng. 25(5), 1148–1161 (2013)

    Article  Google Scholar 

  54. Zhao, M., Jiao, L., Feng, J., Liu, T.: A simplified low rank and sparse graph for semi-supervised learning. Neurocomputing 140, 84–96 (2014)

    Article  Google Scholar 

  55. Zhao, M.-B., Zhang, Z., Chow, T.W.S., Li, B.: A general soft label based linear discriminant analysis for semi-supervised dimensionality reduction. Neural Netw. 55, 83–97 (2014)

    Article  MATH  Google Scholar 

  56. Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Schölkopf, B.: Learning with local and global consistency. In: Advances in Neural Information Processing Systems (NIPS), pp. 321–328 (2003)

    Google Scholar 

  57. Zhu, X.: Semi-supervised learning. In: Encyclopedia of Machine Learning, pp. 892–897. Springer (2010)

    Google Scholar 

  58. Zhu, X., Ghahramani, Z., Lafferty, J.D.: Semi-supervised learning using gaussian fields and harmonic functions. In: International Conference on Machine Learning (ICML), pp. 912–919 (2003)

    Google Scholar 

  59. Zhu, X., Goldberg, A.B., Khot, T.: Some new directions in graph-based semi-supervised learning. In: IEEE International Conference on Multimedia and Expo, pp. 1504–1507 (2009)

    Google Scholar 

  60. Zhuang, L., Gao, H., Lin, Z., Ma, Y., Zhang, X., Yu, N.: Non-negative low rank and sparse graph for semi-supervised learning. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2328–2335 (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Northeastern University, Boston, MA, USA

    Sheng Li & Yun Fu

Authors
  1. Sheng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yun Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Li, S., Fu, Y. (2017). Robust Graph Construction. In: Robust Representation for Data Analytics. Advanced Information and Knowledge Processing. Springer, Cham. https://doi.org/10.1007/978-3-319-60176-2_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-60176-2_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-60175-5

  • Online ISBN: 978-3-319-60176-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation