Open Access
Subscription Access
Open Access
Subscription Access
Graph Neural Network Learning in Large Graphs - A Critical Review
Subscribe/Renew Journal
Graph Neural Networks have been extensively used to learn non-Euclidian structures like graphs. There have been several attempts to improve the training efficiency and to reduce the learning complexity in modelling of large graph datasets. In this paper we have reviewed the approaches which perform convolutions to model large graphs for classification and prediction. We have critically analysed each of these approaches and veracity of their claims of reduced complexity and have reported their shortcomings. We have further analysed the approaches from graph-dataset perspective.
Keywords
Graph Neural Networks, Graph Convolutional Networks, Graph Representation Learning, Large Graph Dataset.
Subscription
Login to verify subscription
User
Font Size
Information
- M. M. Bronstein, J. Bruna, Y. Lecun, A. Szlam and P. Vandergheynst, “Geometric Deep Learning: Going beyond Euclidean Data”, IEEE Signal Processing Magazine, Vol. 34, No. 4, pp. 18-42, 2017.
- W. L. Hamilton, R. Ying and J. Leskovec, “Representation Learning on Graphs: Methods and Applications”, IEEE Data Engineering Bulletin, Vol. 24, No. 2, pp. 1-24, 2017.
- J.B. Lee, R.A. Rossi, S. Kim, N.K. Ahmed and E. Koh, “Attention Models in Graphs: A Survey”, Proceedings of International Conference on Artificial Intelligence, pp. 1-13, 2018.
- P. W. Battaglia, “Relational Inductive Biases, Deep Learning, and Graph Networks”, Proceedings of International Conference on Artificial Intelligence and Machine Learning, pp. 1-14, 2018.
- J. Zhou and S. Hu, “Graph Neural Networks: A Review of Methods and Applications”, Proceedings of International Conference on Machine Learning, pp. 1-22, 2018.
- Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang and P. S. Yu, “A Comprehensive Survey on Graph Neural Networks”, Proceedings of International Conference on Network Embedding and Graph Neural Networks, pp. 1-22, 2019.
- Z. Zhang, P. Cui and W. Zhu, “Deep Learning on Graphs: A Survey”, IEEE Transactions on Knowledge and Data Engineering, Vol. 14, No. 8, pp. 1-24, 2018.
- A. Sperduti and A. Starita, “Supervised Neural Networks for the Classification of Structures”, IEEE Transactions on Neural Networks, Vol. 8, No. 3, pp. 714-735, 1997.
- M. Gori, G. Monfardini and F. Scarselli, “A New Model for Learning in Graph Domains”, Proceedings of International Conference on Neural Networks, Vol. 2, No. 2, pp. 729-734, 2005.
- F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner and G. Monfardini, “The Graph Neural Network Model”, IEEE Transactions on Neural Networks, Vol. 20, No. 1, pp. 61-80, 2009.
- J. Bruna, W. Zaremba, A. Szlam and Y. Le Cun, “Spectral Networks and Locally Connected Networks on Graphs”, Proceedings of International Conference on Computer Vision and Pattern Recognition, pp. 1-14, 2013.
- M. Henaff, J. Bruna and Y. Le Cun, “Deep Convolutional Networks on Graph-Structured Data”, Proceedings of International Conference on Computer Vision and Pattern Recognition, pp. 1-10, 2015.
- M. Defferrard, X. Bresson and P. Vandergheynst, “Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering”, Proceedings of International Conference on Advances in Neural Information Processing Systems, pp. 3844-3852, 2016.
- T.N. Kipf and M. Welling, “Semi-Supervised Classification with Graph Convolutional Networks”, Proceedings of International Conference on Machine Learning, pp. 1-14, 2016.
- R. Levie, F. Monti, X. Bresson and M.M. Bronstein, “CayleyNets: Graph Convolutional Neural Networks with Complex Rational Spectral Filters”, IEEE Transactions on Signal Processing, Vol. 67, No. 1, pp. 97-109, 2019.
- A. Micheli, “Neural Network for Graphs: A Contextual Constructive Approach”, IEEE Transactions on Neural Networks, Vol. 20, No. 3, pp. 498-511, 2009.
- W.L. Hamilton, R. Ying and J. Leskovec, “Inductive Representation Learning on Large Graphs”, Proceedings of International Conference on Advances in Neural Information Processing Systems, pp. 1025-1035, 2017.
- S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory”, Neural Computing, Vol. 9, No. 8, pp. 1735-1780, 1997.
- B. Perozzi, R. Al-Rfou and S. Skiena, “Deep Walk: Online Learning of Social Representations”, Proceedings of International Conference on Knowledge Discovery and Data Mining, pp. 701-710, 2014.
- W.L. Chiang, Y. Li, X. Liu, S. Bengio, S. Si and C. J. Hsieh, “Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks”, Proceedings of International Conference on Knowledge Discovery and Data Mining, pp. 257-266, 2019.
- J. Chen, J. Zhu and L. Song, “Stochastic Training of Graph Convolutional Networks with Variance Reduction”, Proceedings of International Conference on Machine Learning, pp. 1503-1532, 2018.
- M. T. Chen Jie and X. Cao, “FASTGCN : Fast Learning With GCN vis Importance Sampling”, Proceedings of International Conference on Machine Learning, pp. 1-15, 2018.
- W. Huang, T. Zhang, Y. Rong and J. Huang, “Adaptive Sampling Towards Fast Graph Representation Learning”, Proceedings of International Conference on Advances in Neural Information Processing Systems, pp. 4558-4567, 2018.
- P. Velickovic, G. Cucurull, A. Casanova, A. Romero, P. Lio and Y. Bengio, “Graph Attention Networks”, Proceedings of International Conference on Artificial Intelligence, pp. 1-12, 2017.
- F. Fouss, K. Francoisse, L. Yen, A. Pirotte and M. Saerens, “An Experimental Investigation of Kernels on Graphs for Collaborative Recommendation and Semisupervised Classification”, Neural Networks, Vol. 31, pp. 53-72, 2012.
- J. Atwood and D. Towsley, “Diffusion-Convolutional Neural Networks”, Proceedings of International Conference on Advances in Neural Information Processing Systems, pp. 1-15, 2016.
- J. Zhang, X. Shi, J. Xie, H. Ma, I. King and D.Y. Yeung, “GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs”, Proceedings of International Conference on Artificial Intelligence, pp. 339-349, 2018.
- H. Gao, Z. Wang and S. Ji, “Large-Scale Learnable Graph Convolutional Networks”, Proceedings of International Conference on Knowledge Discovery and Data Mining, pp. 1416-1424, 2018.
- R. Ying, R. He, K. Chen, P. Eksombatchai, W.L. Hamilton and J. Leskovec, “Graph Convolutional Neural Networks for Web-Scale Recommender Systems”, Proceedings of International Conference on Knowledge Discovery and Data Mining, pp. 974-983, 2018.
- H. Pei, B. Wei, K. C.C. Chang, Y. Lei and B. Yang, “Geom-GCN: Geometric Graph Convolutional Networks”, Proceedings of International Conference on Artificial Intelligence and Machine Learning, pp. 1-12, 2020.
- V. De, L.J. Tenenbaum and S. Joshua, “A Global Geometric Framework for Nonlinear Dimensionality Reduction”, Proceedings of International Conference on Geometric Structure of High-Dimensional Data and Dimensionality Reduction, pp. 151-180, 2012.
- M. Nickel and D. Kiela, “Poincare Embeddings for Learning Hierarchical Representations”, Proceedings of International Conference on Advances in Neural Information Processing Systems, pp. 6339-6348, 2017.
- L.F.R. Ribeiro, P.H.P. Saverese and D.R. Figueiredo, “Struc2vec: Learning Node Representations from Structural Identity”, Proceedings of International Conference on Knowledge Discovery and Data Mining, pp. 385-394, 2017.
- Y. Liu, “Community Enhanced Graph Convolutional Networks”, Pattern Recognitions Letters, Vol. 138, pp. 462-468, 2020.
- B. Shmueli, “Multi-Class Metrics Made Simple, Part II: the F1-Score”, Available at https://towardsdatascience.com/multi-class-metrics-made-simple-part-ii-the-f1-score-ebe8b2c2ca1, Accessed at 2019.
- Ryan Rossi and Nesreen Ahmed, “Network Statistics”, Available at http://networkrepository.com/.
Abstract Views: 348
PDF Views: 1