Abstract
Statistical Relational Learning (SRL) methods have shown that classification accuracy can be improved by integrating relations between samples. Techniques such as iterative classification or relaxation labeling achieve this by propagating information between related samples during the inference process. When only a few samples are labeled and connections between samples are sparse, collective inference methods have shown large improvements over standard feature-based ML methods. However, in contrast to feature based ML, collective inference methods require complex inference procedures and often depend on the strong assumption of label consistency among related samples. In this paper, we introduce new relational features for standard ML methods by extracting information from direct and indirect relations. We show empirically on three standard benchmark datasets that our relational features yield results comparable to collective inference methods. Finally we show that our proposal outperforms these methods when additional information is available.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bernstein, A., Clearwater, S., Provost, F.: The relational vector-space model and industry classification. In: IJCAI Workshop, vol. 266 (2003)
Chakrabarti, S., Dom, B., Indyk, P.: Enhanced hypertext categorization using hyperlinks. ACM SIGMOD Record 27(2), 307–318 (1998)
Crane, R., McDowell, L.: Investigating markov logic networks for collective classification. In: ICAART (1), pp. 5–15 (2012)
Gallagher, B., Eliassi-Rad, T.: Leveraging Label-Independent Features for Classification in Sparsely Labeled Networks: An Empirical Study. In: Giles, L., Smith, M., Yen, J., Zhang, H. (eds.) SNAKDD 2008. LNCS, vol. 5498, pp. 1–19. Springer, Heidelberg (2010)
Gallagher, B., Tong, H., Eliassi-Rad, T., Faloutsos, C.: Using ghost edges for classification in sparsely labeled networks. In: KDD, pp. 256–264 (2008)
Graepel, T., Candela, J.Q., Borchert, T., Herbrich, R.: Web-scale bayesian click-through rate prediction for sponsored search advertising in microsoft’s bing search engine. In: ICML, pp. 13–20 (2010)
Jensen, D., Neville, J., Gallagher, B.: Why collective inference improves relational classification. In: KDD, pp. 593–598 (2004)
Karypis, G., Kumar, V.: A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM Journal on Scientific Computing 20(1), 359–392 (1998)
Kou, Z., Cohen, W.W.: Stacked graphical models for efficient inference in markov random fields. In: SDM, pp. 533–538. SIAM (2007)
Liu, J., Chen, J., Ye, J.: Large-scale sparse logistic regression. In: KDD, pp. 547–556. ACM (2009)
Lu, Q., Getoor, L.: Link-based classification. In: ICML, vol. 3, pp. 496–503 (2003)
Macskassy, S.A.: Improving learning in networked data by combining explicit and mined links. AAAI 22, 590–595 (2007)
Macskassy, S.A., Provost, F.: A simple relational classifier. In: KDD-Workshop, pp. 64–76 (2003)
Macskassy, S.A., Provost, F.: Classification in networked data: A toolkit and a univariate case study. JMLR 8, 935–983 (2007)
McDowell, L.K., Gupta, K.M., Aha, D.W.: Cautious collective classification. JMLR 10, 2777–2836 (2009)
Mukherjee, I., Canini, K., Frongillo, R., Singer, Y.: Parallel Boosting with Momentum. In: Blockeel, H., Kersting, K., Nijssen, S., Železný, F. (eds.) ECML PKDD 2013, Part III. LNCS, vol. 8190, pp. 17–32. Springer, Heidelberg (2013)
Murphy, K.P.: Machine learning: A probabilistic perspective. The MIT Press (2012)
Neville, J., Jensen, D.: Iterative classification in relational data. In: Proc. AAAI-2000 Workshop on Learning Statistical Models from Relational Data, pp. 13–20 (2000)
Neville, J., Jensen, D.: Collective classification with relational dependency networks. In: UAI, pp. 77–91 (2003)
Neville, J., Jensen, D., Friedland, L., Hay, M.: Learning relational probability trees. In: KDD, pp. 625–630 (2003)
Neville, J., Jensen, D., Gallagher, B.: Simple estimators for relational bayesian classifiers. In: ICDM, pp. 609–612 (2003)
Pan, J.Y., Yang, H.J., Faloutsos, C., Duygulu, P.: Automatic multimedia cross-modal correlation discovery. In: KDD, pp. 653–658. ACM (2004)
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: Machine learning in Python. JMLR 12, 2825–2830 (2011)
Perlich, C., Provost, F.: Distribution-based aggregation for relational learning with identifier attributes. Machine Learning 62(1), 65–105 (2006)
Perlich, C., Provost, F.: Aggregation-based feature invention and relational concept classes. In: KDD, pp. 167–176. ACM (2003)
Preisach, C., Schmidt-Thieme, L.: Relational ensemble classification. In: ICDM, pp. 499–509 (2006)
Rendle, S.: Scaling factorization machines to relational data. In: VLDB. vol. 6, pp. 337–348. VLDB Endowment (2013)
Richardson, M., Domingos, P.: Markov logic networks. Machine Learning 62, 107–136 (2006)
Saar-Tsechansky, M., Provost, F.: Handling missing values when applying classification models. JMLR (2007)
Sen, P., Namata, G.M., Bilgic, M., Getoor, L., Gallagher, B., Eliassi-Rad, T.: Collective classification in network data. AI Magazine 29(3), 93–106 (2008)
Taskar, B., Segal, E., Koller, D.: Probabilistic classification and clustering in relational data. IJCAI 17, 870–878 (2001)
Tong, H., Faloutsos, C., Pan, J.Y.: Fast random walk with restart and its applications. In: ICDM, pp. 613–622 (2006)
Xiang, R., Neville, J.: Understanding propagation error and its effect on collective classification. In: ICDM, pp. 834–843. IEEE (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Bayer, I., Nagel, U., Rendle, S. (2015). Graph Based Relational Features for Collective Classification. In: Cao, T., Lim, EP., Zhou, ZH., Ho, TB., Cheung, D., Motoda, H. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2015. Lecture Notes in Computer Science(), vol 9078. Springer, Cham. https://doi.org/10.1007/978-3-319-18032-8_35
Download citation
DOI: https://doi.org/10.1007/978-3-319-18032-8_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-18031-1
Online ISBN: 978-3-319-18032-8
eBook Packages: Computer ScienceComputer Science (R0)