Abstract
Semi-supervised learning has witnessed increasing interest in the past decade. One common assumption behind semi-supervised learning is that the data labels should be sufficiently smooth with respect to the intrinsic data manifold. Recent research has shown that the features also lie on a manifold. Moreover, there is a duality between data points and features, that is, data points can be classified based on their distribution on features, while features can be classified based on their distribution on the data points. However, existing semi-supervised learning methods neglect these points. Based on the above observations, in this paper, we present a dual regularization, which consists of two graph regularizers and a co-clustering type regularizer. In detail, the two graph regularizers consider the geometric structure of the data points and the features respectively, while the co-clustering type regularizer takes into account the duality between data points and features. Furthermore, we propose a novel transductive classification framework based on dual regularization, which can be solved by alternating minimization algorithm and its convergence is theoretically guaranteed. Experiments on benchmark semi-supervised learning data sets demonstrate that the proposed methods outperform many state of the art transductive classification methods.
Chapter PDF
Similar content being viewed by others
Keywords
- Nonnegative Matrix Factorization
- Feature Graph
- Local Linear Embedding
- Graph Regularizer
- Transductive Learning
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Chapelle, O., Schölkopf, B., Zien, A. (eds.): Semi-Supervised Learning. MIT Press, Cambridge (2006)
Zhu, X.: Semi-supervised learning literature survey. Technical report, Computer Sciences, University of Wisconsin-Madison (2008)
Zhu, X., Ghahramani, Z., Lafferty, J.D.: Semi-supervised learning using gaussian fields and harmonic functions. In: ICML, pp. 912–919 (2003)
Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Schölkopf, B.: Learning with local and global consistency. In: NIPS (2003)
Wang, F., Zhang, C.: Label propagation through linear neighborhoods. In: ICML, pp. 985–992 (2006)
Wu, M., Schölkopf, B.: Transductive classification via local learning regularization. In: AISTATS, pp. 628–635 (March 2007)
Wang, F., Li, T., Wang, G., Zhang, C.: Semi-supervised classification using local and global regularization. In: AAAI, pp. 726–731 (2008)
Belkin, M., Niyogi, P., Sindhwani, V.: Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research 7, 2399–2434 (2006)
Smola, A.J., Kondor, R.: Kernels and regularization on graphs. In: Schölkopf, B., Warmuth, M.K. (eds.) COLT/Kernel 2003. LNCS (LNAI), vol. 2777, pp. 144–158. Springer, Heidelberg (2003)
Gu, Q., Zhou, J.: Local learning regularized nonnegative matrix factorization. In: IJCAI (2009)
Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)
Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation 15, 1373–1396 (2003)
Sandler, T., Blitzer, J., Talukdar, P., Pereira, F.: Regularized learning with networks of features. In: NIPS (2008)
Gu, Q., Zhou, J.: Co-clustering on manifolds. In: KDD (2009)
Dhillon, I.S.: Co-clustering documents and words using bipartite spectral graph partitioning. In: KDD, pp. 269–274 (2001)
Dhillon, I.S., Mallela, S., Modha, D.S.: Information-theoretic co-clustering. In: KDD, pp. 89–98 (2003)
Ding, C.H.Q., Li, T., Peng, W., Park, H.: Orthogonal nonnegative matrix t-factorizations for clustering. In: KDD, pp. 126–135 (2006)
Sindhwani, V., Hu, J., Mojsilovic, A.: Regularized co-clustering with dual supervision. In: NIPS, pp. 556–562 (2008)
Sindhwani, V., Melville, P.: Document-word co-regularization for semi-supervised sentiment analysis. In: ICDM, pp. 1025–1030 (2008)
Li, T., Ding, C.H.Q., Zhang, Y., Shao, B.: Knowledge transformation from word space to document space. In: SIGIR, pp. 187–194 (2008)
Chung, F.R.K.: Spectral Graph Theory. American Mathematical Society (February 1997)
Boyd, S., Vandenberghe, L.: Convex optimization. Cambridge University Press, Cambridge (2004)
Ding, C.H., Li, T., Jordan, M.I.: Convex and semi-nonnegative matrix factorizations. IEEE Transactions on Pattern Analysis and Machine Intelligence 99(1) (2008)
Lee, D.D., Seung, H.S.: Algorithms for non-negative matrix factorization. In: NIPS, pp. 556–562 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gu, Q., Zhou, J. (2009). Transductive Classification via Dual Regularization. In: Buntine, W., Grobelnik, M., Mladenić, D., Shawe-Taylor, J. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2009. Lecture Notes in Computer Science(), vol 5781. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04180-8_46
Download citation
DOI: https://doi.org/10.1007/978-3-642-04180-8_46
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04179-2
Online ISBN: 978-3-642-04180-8
eBook Packages: Computer ScienceComputer Science (R0)