Representation Learning on Graphs
The primary challenge of applying machine learning in graph theory is finding a way to represent, or encode, graph structure so that it can be easily exploited by machine learning models. Traditionally, machine learning approaches relied on user-defined heuristics to extract features encoding structural information about a graph. However, recent years have seen a surge in approaches that automatically learn to encode graph structure into low-dimensional embeddings, using techniques based on deep learning and nonlinear dimensionality reduction. In this chapter, we will look at a review of key advancements in this area of representation learning on graphs, including matrix factorization-based methods, random-walk based algorithms, and graph convolutional networks. We will also look at methods to embed individual nodes as well as approaches to embed entire (sub)graphs.
- 1.Ahmed, Amr, Nino Shervashidze, Shravan Narayanamurthy, Vanja Josifovski, and Alexander J. Smola. Distributed large-scale natural graph factorization. In Proceedings of the 22nd international conference on World Wide Web, 37–48. ACM, 2013.Google Scholar
- 2.Bronstein, Michael, M., Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. 2017. Geometric deep learning: Going beyond Euclidean data. IEEE Signal Processing Magazine 34 (4): 18–42.Google Scholar
- 3.Bruna, Joan, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. 2013. Spectral networks and locally connected networks on graphs. arXiv:1312.6203.
- 4.Cao, Shaosheng, Wei Lu, and Qiongkai Xu. 2015. Grarep: Learning graph representations with global structural information. In Proceedings of the 24th ACM international on conference on information and knowledge management, 891–900. ACM.Google Scholar
- 5.Cao, Shaosheng, Wei Lu, and Qiongkai Xu. 2016. Deep neural networks for learning graph representations. In AAAI, 1145–1152.Google Scholar
- 6.Chang, Shiyu, Wei Han, Jiliang Tang, Guo-Jun Qi, Charu C. Aggarwal, and Thomas S. Huang. 2015. Heterogeneous network embedding via deep architectures. In Proceedings of the 21st ACM SIGKDD international conference on knowledge discovery and data mining, 119–128. ACM.Google Scholar
- 7.Chen, Haochen, Bryan Perozzi, Yifan Hu, and Steven Skiena. 2017. Harp: Hierarchical representation learning for networks. arXiv:1706.07845.
- 8.Dai, Hanjun, Bo Dai, and Le Song. 2016. Discriminative embeddings of latent variable models for structured data. In International conference on machine learning, 2702–2711.Google Scholar
- 9.Defferrard, Michaël, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in neural information processing systems, 3844–3852.Google Scholar
- 10.Dong, Yuxiao, Nitesh V. Chawla, and Ananthram Swami. 2017. metapath2vec: Scalable representation learning for heterogeneous networks. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, 135–144. ACM.Google Scholar
- 11.Donnat, Claire, Marinka Zitnik, David Hallac, and Jure Leskovec. 2017. Spectral graph wavelets for structural role similarity in networks. arXiv:1710.10321.
- 12.Duvenaud, David K., Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Timothy Hirzel, Alán Aspuru-Guzik, and Ryan P. Adams. 2015. Convolutional networks on graphs for learning molecular fingerprints. In Advances in neural information processing systems, 2224–2232.Google Scholar
- 13.Grover, Aditya, and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 855–864. ACM.Google Scholar
- 14.Hamilton, Will, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Advances in neural information processing systems, 1025–1035.Google Scholar
- 15.Hamilton, William L., Rex Ying, and Jure Leskovec. 2017. Representation learning on graphs: Methods and applications. arXiv:1709.05584.
- 16.Hinton, Geoffrey E., and Ruslan R Salakhutdinov. 2006. Reducing the dimensionality of data with neural networks. Science 313 (5786): 504–507.Google Scholar
- 17.Hochreiter, Sepp, and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation 9 (8): 1735–1780.Google Scholar
- 18.Kipf, Thomas N., and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv:1609.02907.
- 19.Kipf, Thomas N., and Max Welling. 2016. Variational graph auto-encoders. arXiv:1611.07308.
- 20.Li, Yujia, Daniel Tarlow, Marc Brockschmidt, and Richard Zemel. 2015. Gated graph sequence neural networks. arXiv:1511.05493.
- 21.Nickel, Maximilian, Kevin Murphy, Volker Tresp, and Evgeniy Gabrilovich. 2016. A review of relational machine learning for knowledge graphs. Proceedings of the IEEE 104 (1): 11–33.Google Scholar
- 22.Ou, Mingdong, Peng Cui, Jian Pei, Ziwei Zhang, and Wenwu Zhu. 2016. Asymmetric transitivity preserving graph embedding. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 1105–1114. ACM.Google Scholar
- 23.Perozzi, Bryan, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on knowledge discovery and data mining, 701–710. ACM.Google Scholar
- 24.Pham, Trang, Truyen Tran, Dinh Q. Phung, and Svetha Venkatesh. 2017. Column networks for collective classification. In AAAI, 2485–2491.Google Scholar
- 25.Ribeiro, Leonardo FR., Pedro HP. Saverese, and Daniel R. Figueiredo. 2017. struc2vec: Learning node representations from structural identity. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, 385–394. ACM.Google Scholar
- 26.Scarselli, Franco, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. 2009. The graph neural network model. IEEE Transactions on Neural Networks 20 (1): 61–80.Google Scholar
- 27.Schlichtkrull, Michael, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. 2017. Modeling relational data with graph convolutional networks. arXiv:1703.06103.
- 28.Tang, Jian, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. Line: Large-scale information network embedding. In Proceedings of the 24th international conference on World Wide Web, 1067–1077. (International World Wide Web Conferences Steering Committee).Google Scholar
- 29.van den Berg, Rianne, Thomas N. Kipf, and Max Welling. 2017. Graph convolutional matrix completion. Statistics 1050: 7.Google Scholar
- 30.Wang, Daixin, Peng Cui, and Wenwu Zhu. 2016. Structural deep network embedding. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 1225–1234. ACM.Google Scholar
- 31.Zitnik, Marinka, and Jure Leskovec. 2017. Predicting multicellular function through multi-layer tissue networks. Bioinformatics 33 (14): i190–i198.Google Scholar