Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Data Augmentation for Deep Graph Learning

Data Augmentation for Deep Graph Learning Graph neural networks, a powerful deep learning tool to model graph-structured data, have demonstrated remarkable performance on numerous graph learning tasks. To address the data noise and data scarcity issues in deep graph learning, the research on graph data augmentation has intensified lately. However, conventional data augmentation methods can hardly handle graph-structured data which is defined in non-Euclidean space with multi-modality. In this survey, we formally formulate the problem of graph data augmentation and further review the representative techniques and their applications in different deep graph learning problems. Specifically, we first propose a taxonomy for graph data augmentation techniques and then provide a structured review by categorizing the related work based on the augmented information modalities. Moreover, we summarize the applications of graph data augmentation in two representative problems in data-centric deep graph learning: (1) reliable graph learning which focuses on enhancing the utility of input graph as well as the model capacity via graph data augmentation; and (2) low-resource graph learning which targets on enlarging the labeled training data scale through graph data augmentation. For each problem, we also provide a hierarchical problem taxonomy and review the existing literature related to graph data augmentation. Finally, we point out promising research directions and the challenges in future research. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM SIGKDD Explorations Newsletter Association for Computing Machinery

Data Augmentation for Deep Graph Learning

Download PDF
17 pages

Loading next page...
 
/lp/association-for-computing-machinery/data-augmentation-for-deep-graph-learning-mwJlPlMtfl
Publisher
Association for Computing Machinery
Copyright
Copyright © 2022 Copyright is held by the owner/author(s)
ISSN
1931-0145
eISSN
1931-0153
DOI
10.1145/3575637.3575646
Publisher site
See Article on Publisher Site

Abstract

Graph neural networks, a powerful deep learning tool to model graph-structured data, have demonstrated remarkable performance on numerous graph learning tasks. To address the data noise and data scarcity issues in deep graph learning, the research on graph data augmentation has intensified lately. However, conventional data augmentation methods can hardly handle graph-structured data which is defined in non-Euclidean space with multi-modality. In this survey, we formally formulate the problem of graph data augmentation and further review the representative techniques and their applications in different deep graph learning problems. Specifically, we first propose a taxonomy for graph data augmentation techniques and then provide a structured review by categorizing the related work based on the augmented information modalities. Moreover, we summarize the applications of graph data augmentation in two representative problems in data-centric deep graph learning: (1) reliable graph learning which focuses on enhancing the utility of input graph as well as the model capacity via graph data augmentation; and (2) low-resource graph learning which targets on enlarging the labeled training data scale through graph data augmentation. For each problem, we also provide a hierarchical problem taxonomy and review the existing literature related to graph data augmentation. Finally, we point out promising research directions and the challenges in future research.

Journal

ACM SIGKDD Explorations Newsletter – Association for Computing Machinery

Published: Dec 5, 2022

References

  • On the bottleneck of graph neural networks and its practical implications
    Alon, U.; Yahav, E.
  • Local graph partitioning using pagerank vectors
    Andersen, R.; Chung, F.; Lang, K.
  • Neural networks and their applications
    Bishop, C. M.
  • Combining labeled and unlabeled data with co-training
    Blum, A.; Mitchell, T.
  • Regularizing graph neural networks via consistencydiversity graph augmentations
    Bo, D.; Hu, B.; X.Wang, ; Zhang, Z.; Shi, C.; Zhou, J.
  • Scaling graph neural networks with approximate pagerank
    Bojchevski, A.; Klicpera, J.; Perozzi, B.; Kapoor, A.; Blais, M.; R´ozemberczki, B.; Lukasik, M.; G¨unnemann, S.
  • Improving graph neural network expressivity via subgraph isomorphism counting
    Bouritsas, G.; Frasca, F.; Zafeiriou, S. P.; Bronstein, M.
  • Graph coarsening with neural networks
    Cai, C.; D.Wang, ; Y.Wang,
  • Measuring and relieving the over-smoothing problem for graph neural networks from the topological view
    Chen, D.; Lin, Y.; Li, W.; Li, P.; Zhou, J.; Sun, X.
  • Graph masked autoencoder
    Chen, H.; Zhang, S.; Xu, G.
  • Fastgcn: Fast learning with graph convolutional networks via importance sampling
    Chen, J.; Ma, T.; Xiao, C.
  • On graph neural networks versus graph-augmented mlps
    Chen, L.; Chen, Z.; Bruna, J.
  • Trading personalization for accuracy: Data debugging in collaborative filtering
    Chen, L.; Yao, Y.; Xu, F.; Xu, M.; Tong, H.
  • Scalable graph neural networks via bidirectional propagation
    Chen, M.; Wei, Z.; Ding, B.; Li, Y.; Yuan, Y.; Du, X.; Wen, J.-R.
  • Learning on attribute-missing graphs
    Chen, X.; Chen, S.; Yao, J.; Zheng, H.; Zhang, Y.; Tsang, I. W.
  • Iterative deep graph learning for graph neural networks: Better and robust node embeddings
    Chen, Y.; Wu, L.; Zaki, M.
  • Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks
    Chiang, W.-L.; Liu, X.; Si, S.; Li, Y.; Bengio, S.; Hsieh, C.-J.
  • Nrgnn: Learning a label noise-resistant graph neural network on sparsely and noisily labeled graphs
    Dai, E.; Aggarwal, C.; Wang, S.
  • Batch virtual adversarial training for graph convolutional networks
    Deng, Z.; Dong, Y.; Zhu, J.
  • Deep anomaly detection on attributed networks
    Ding, K.; Li, J.; Bhanushali, R.; Liu, H.
  • Meta propagation networks for graph few-shot semi-supervised learning
    Ding, K.; Wang, J.; Caverlee, J.; Liu, H.
  • Be more with less: Hypergraph attention networks for inductive text classification
    Ding, K.; Wang, J.; Li, J.; Li, D.; Liu, H.
  • Graph prototypical networks for few-shot learning on attributed networks
    Ding, K.; Wang, J.; Li, J.; Shu, K.; Liu, C.; Liu, H.
  • Structural and semantic contrastive learning for self-supervised node representation learning
    Ding, K.; Wang, Y.; Yang, Y.; Liu, H.
  • On the equivalence of decoupled graph convolution network and label propagation
    Dong, H.; Chen, J.; Feng, F.; He, X.; Bi, S.; Ding, Z.; Cui, P.
  • All you need is low (rank) defending against adversarial attacks on graphs
    Entezari, N.; Al-Sayouri, S. A.; Darvishzadeh, A.; Papalexakis, E. E.
  • Graph adversarial training: Dynamically regularizing based on graph structure
    Feng, F.; He, X.; Tang, J.; Chua, T.-S.
  • Adversarial graph contrastive learning with information regularization
    Feng, S.; Jing, B.; Zhu, Y.; Tong, H.
  • Grand+: Scalable graph random neural networks
    Feng, W.; Dong, Y.; Huang, T.; Yin, Z.; Cheng, X.; Kharlamov, E.; Tang, J.
  • Graph random neural networks for semi-supervised learning on graphs
    Feng, W.; Zhang, J.; Dong, Y.; Han, Y.; Luan, H.; Xu, Q.; Yang, Q.; Kharlamov, E.; Tang, J.
  • Learning discrete structures for graph neural networks
    Franceschi, L.; Niepert, M.; Pontil, M.; He, X.
  • Training robust graph neural networks with topology adaptive edge dropping
    Gao, Z.; Bhattacharya, S.; Zhang, L.; Blum, R. S.; Ribeiro, A.; Sadler, B. M.
  • Neural message passing for quantum chemistry
    Gilmer, J.; Schoenholz, S. S.; Riley, P. F.; Vinyals, O.; Dahl, G. E.
  • Explaining and harnessing adversarial examples
    Goodfellow, I. J.; Shlens, J.; Szegedy, C.
  • ifmixup: Towards intrusion-free graph mixup for graph classification
    Guo, H.; Mao, Y.
  • Inductive representation learning on large graphs
    Hamilton, W. L.; Ying, R.; Leskovec, J.
  • G-mixup: Graph data augmentation for graph classification
    Han, X.; Jiang, Z.; Liu, N.; Hu, X.
  • Contrastive multiview representation learning on graphs
    Hassani, K.; Khasahmadi, A. H.
  • Learning graph augmentations to learn graph representations
    Hassani, K.; Khasahmadi, A. H.
  • Scalable consistency training for graph neural networks via self-ensemble self-distillation
    Hawkins, C.; Ioannidis, V. N.; Adeshina, S.; Karypis, G.
  • Analyzing heterogeneous networks with missing attributes by unsupervised contrastive learning
    He, D.; Liang, C.; Huo, C.; Feng, Z.; Jin, D.; Yang, L.; Zhang, W.
  • Fraudar: Bounding graph fraud in the face of camouflage
    Hooi, B.; Song, H. A.; Beutel, A.; Shah, N.; Shin, K.; Faloutsos, C.
  • Graphmae: Self-supervised masked graph autoencoders
    Hou, Z.; Liu, X.; Dong, Y.; Wang, C.; Tang, J.
  • Gpt-gnn: Generative pre-training of graph neural networks
    Hu, Z.; Dong, Y.; Wang, K.; Chang, K.-W.; Sun, Y.
  • Pre-training graph neural networks for generic structural feature extraction
    Hu, Z.; Fan, C.; Chen, T.; Chang, K.-W.; Sun, Y.
  • Scaling up graph neural networks via graph coarsening
    Huang, Z.; Zhang, S.; Xi, C.; Liu, T.; Zhou, M.
  • Collaborative graph convolutional networks: Unsupervised learning meets semisupervised learning
    Hui, B.; Zhu, P.; Hu, Q.
  • Maeda, and M
    Ishiguro, K.; S.-i,
  • Scaling personalized web search
    Jeh, G.; J.Widom,
  • Sub-graph contrast for scalable self-supervised graph representation learning
    Jiao, Y.; Xiong, Y.; Zhang, J.; Zhang, Y.; Zhang, T.; Zhu, Y.
  • Heterogeneous graph neural network via attribute completion
    Jin, D.; Huo, C.; Liang, C.; Yang, L.
  • Multi-scale contrastive siamese networks for self-supervised graph representation learning
    Jin, M.; Zheng, Y.; Li, Y.-F.; Gong, C.; Zhou, C.; Pan, S.
  • Graph structure learning for robust graph neural networks
    Jin, W.; Ma, Y.; Liu, X.; Tang, X.; S.Wang, ; Tang, J.
  • Condensing graphs via one-step gradient matching
    Jin, W.; Tang, X.; Jiang, H.; Li, Z.; Zhang, D.; Tang, J.; Yin, B.
  • Graph condensation for graph neural networks
    Jin, W.; Zhao, L.; Zhang, S.; Liu, Y.; Tang, J.; Shah, N.
  • Graph coarsening with preserved spectral properties
    Jin, Y.; Loukas, A.; JaJa, J.
  • Differentiable graph module (dgm) for graph convolutional networks
    Kazi, A.; Cosmo, L.; Ahmadi, S.-A.; Navab, N.; Bronstein, M.
  • Variational graph autoencoders
    Kipf, T. N.; Welling, M.
  • Semi-supervised classification with graph convolutional networks
    Kipf, T. N.; Welling, M.
  • Predict then propagate: Graph neural networks meet personalized pagerank
    Klicpera, J.; Bojchevski, A.; G¨unnemann, S.
  • Diffusion improves graph learning
    Klicpera, J.; Weißenberger, S.; G¨unnemann, S.
  • Diffusion kernels on graphs and other discrete structures
    Kondor, R. I.; Lafferty, J.
  • Flag: Adversarial data augmentation for graph neural networks
    Kong, K.; Li, G.; Ding, M.; Wu, Z.; Zhu, C.; Ghanem, B.; Taylor, G.; Goldstein, T.
  • Augmentation-free selfsupervised learning on graphs
    Lee, N.; Lee, J.; Park, C.
  • Learning graph-level representation for drug discovery
    Li, J.; Cai, D.; He, X.
  • Distance encoding: Design provably more powerful neural networks for graph representation learning
    Li, P.; Wang, Y.; Wang, H.; Leskovec, J.
  • Deeper insights into graph convolutional networks for semi-supervised learning
    Li, Q.; Han, Z.; Wu, X.-M.
  • Co-gcn for multi-view semi-supervised learning
    Li, S.; Li, W.-T.; Wang, W.
  • Graph neural network with curriculum learning for imbalanced node classification
    Li, X.; Wen, L.; Deng, Y.; Feng, F.; Hu, X.; Wang, L.; Fan, Z.
  • Local augmentation for graph neural networks
    Liu, S.; Dong, H.; Li, L.; Xu, T.; Rong, Y.; Zhao, P.; Huang, J.; Wu, D.
  • Graph neural networks with adaptive residual
    Liu, X.; Ding, J.; Jin, W.; Xu, H.; Ma, Y.; Liu, Z.; Tang, J.
  • Graph self-supervised learning: A survey
    Liu, Y.; Jin, M.; Pan, S.; Zhou, C.; Zheng, Y.; Xia, F.; Yu, P.
  • Gatsmote: Improving imbalanced node classification on graphs via attention and homophily
    Liu, Y.; Zhang, Z.; Liu, Y.; Zhu, Y.
  • Cf-gnnexplainer: Counterfactual explanations for graph neural networks
    Lucic, A.; Hoeve, M. A. Ter; Tolomei, G.; De Rijke, M.; Silvestri, F.
  • Learning to drop: Robust graph neural network via topological denoising
    Luo, D.; Cheng, W.; Yu, W.; Zong, B.; Ni, J.; Chen, H.; Zhang, X.
  • Automated data augmentations for graph classification
    Luo, Y.; McThrow, M.; Au, W. Y.; Komikado, T.; Uchino, K.; Maruhash, K.; Ji, S.
  • Masked graph modeling for molecule generation
    Mahmood, O.; Mansimov, E.; Bonneau, R.; Cho, K.
  • Simple unsupervised graph representation learning
    Mo, Y.; Peng, L.; Xu, J.; Shi, X.; Zhu, X.
  • Weisfeiler and leman go neural: Higher-order graph neural networks
    Morris, C.; Ritzert, M.; Fey, M.; Hamilton, W. L.; Lenssen, J. E.; Rattan, G.; Grohe, M.
  • The pagerank citation ranking: Bringing order to the web
    Page, L.; Brin, S.; Motwani, R.; Winograd, T.
  • Metropolis-hastings data augmentation for graph neural networks
    Park, H.; Lee, S.; Kim, S.; Park, J.; Jeong, J.; Kim, K.-M.; Ha, J.-W.; Kim, H. J.
  • Graph transplant: Node saliency-guided graph mixup with local structure preservation
    Park, J.; Shim, H.; Yang, E.
  • Graph classification via deep learning with virtual nodes
    Pham, T.; Tran, T.; Dam, H.; Venkatesh, S.
  • Gcc: Graph contrastive coding for graph neural network pre-training
    Qiu, J.; Chen, Q.; Dong, Y.; Zhang, J.; Yang, H.; Ding, M.; Wang, K.; Tang, J.
  • Dropedge: Towards deep graph convolutional networks on node classification
    Rong, Y.; Huang, W.; Xu, T.; Huang, J.
  • Sign: Scalable inception graph neural networks
    Rossi, E.; Frasca, F.; Chamberlain, B.; Eynard, D.; Bronstein, M.; Monti, F.
  • On the unreasonable effectiveness of feature propagation in learning on graphs with missing node features
    Rossi, E.; Kenlay, H.; Gorinova, M. I.; Chamberlain, B. P.; Dong, X.; Bronstein, M.
  • Random features strengthen graph neural networks
    Sato, R.; Yamada, M.; Kashima, H.
  • A survey on image data augmentation for deep learning
    Shorten, C.; Khoshgoftaar, T. M.
  • Multi-stage self-supervised learning for graph convolutional networks on graphs with few labeled nodes
    Sun, K.; Lin, Z.; Zhu, Z.
  • Adversarial attack and defense on graph data: A survey
    Sun, L.; Dou, Y.; Yang, C.; J.Wang, ; Yu, P. S.; He, L.; Li, B.
  • Breaking the limit of graph neural networks by improving the assortativity of graphs with local mixing patterns
    Suresh, S.; Budde, V.; Neville, J.; Li, P.; Ma, J.
  • Adversarial graph augmentation to improve graph contrastive learning
    Suresh, S.; Li, P.; Hao, C.; Neville, J.
  • Graph convolutional networks for graphs containing missing features
    Taguchi, H.; Liu, X.; Murata, T.
  • Mgae: Masked autoencoders for self-supervised learning on graphs
    Tan, Q.; Liu, N.; Huang, X.; Chen, R.; Choi, S.-H.; Hu, X.
  • Largescale representation learning on graphs via bootstrapping
    Thakoor, S.; Tallec, C.; Azar, M. G.; Azabou, M.; Dyer, E. L.; Munos, R.; Velickovi´c, P.; Valko, M.
  • The information bottleneck method
    Tishby, N.; Pereira, F. C.; Bialek, W.
  • Understanding oversquashing and bottlenecks on graphs via curvature
    Topping, J.; Di Giovanni, F.; Chamberlain, B. P.; Dong, X.; Bronstein, M. M.
  • Deep graph infomax
    Velickovic, P.; Fedus, W.; Hamilton, W. L.; Li'o, P.; Bengio, Y.; Hjelm, R. D.
  • Manifold mixup: Better representations by interpolating hidden states
    Verma, V.; Lamb, A.; Beckham, C.; Najafi, A.; Mitliagkas, I.; Lopez-Paz, D.; Bengio, Y.
  • Graphmix: Improved training of gnns for semi-supervised learning
    Verma, V.; Qu, M.; Kawaguchi, K.; Lamb, A.; Bengio, Y.; Kannala, J.; Tang, J.
  • Next-item recommendation with sequential hypergraphs
    Wang, J.; Ding, K.; Hong, L.; Liu, H.; Caverlee, J.
  • Graph structure estimation neural networks
    Wang, R.; Mou, S.; Wang, X.; Xiao, W.; Ju, Q.; Shi, C.; Xie, X.
  • A dual-branch graph convolutional network on imbalanced node classification
    Wang, X.; Chen, J.
  • Graphdefense: Towards robust graph convolutional networks
    Wang, X.; Liu, X.; Hsieh, C.-J.
  • Distance-wise prototypical graph neural network in node imbalance classification
    Wang, Y.; Aggarwal, C.; Derr, T.
  • Mixup for node and graph classification
    Wang, Y.; Wang, W.; Liang, Y.; Cai, Y.; Hooi, B.
  • Nodeaug: Semi-supervised node classification with data augmentation
    Wang, Y.; Wang, W.; Liang, Y.; Cai, Y.; Liu, J.; Hooi, B.
  • From decoupling to reconstruction: A robust graph neural network against topology attacks
    Wei, X.; Li, Y.; Qin, X.; Xu, X.; Li, X.; Liu, M.
  • Simplifying graph convolutional networks
    Wu, F.; Souza, A.; Zhang, T.; Fifty, C.; Yu, T.; Weinberger, K.
  • Adversarial examples for graph data: deep insights into attack and defense
    Wu, H.; Wang, C.; Tyshetskiy, Y.; Docherty, A.; Lu, K.; Zhu, L.
  • Graphmixup: Improving class-imbalanced node classification on graphs by self-supervised context prediction
    Wu, L.; Lin, H.; Gao, Z.; Tan, C.; Li, S.
  • Selfsupervised learning on graphs: Contrastive, generative, or predictive
    Wu, L.; Lin, H.; Tan, C.; Gao, Z.; Li, S. Z.
  • Graph information bottleneck
    Wu, T.; Ren, H.; Li, P.; Leskovec, J.
  • A quest for structure: jointly learning the graph structure and semisupervised classification
    Wu, X.; Zhao, L.; Akoglu, L.
  • A comprehensive survey on graph neural networks
    Wu, Z.; Pan, S.; Chen, F.; Long, G.; Zhang, C.; Philip, S. Y.
  • Simgrace: A simple framework for graph contrastive learning without data augmentation
    Xia, J.; Wu, L.; Chen, J.; Hu, B.; Li, S. Z.
  • Unsupervised data augmentation for consistency training
    Xie, Q.; Dai, Z.; Hovy, E.; Luong, T.; Le, Q.
  • Selfsupervised learning of graph neural networks: A unified review
    Xie, Y.; Xu, Z.; Zhang, J.; Wang, Z.; Ji, S.
  • How powerful are graph neural networks? In ICLR, 2018
    Xu, K.; Hu, W.; Leskovec, J.; Jegelka, S.
  • Graph sanitation with application to node classification
    Xu, Z.; Du, B.; Tong, H.
  • Topology optimization based graph convolutional network
    Yang, L.; Kang, Z.; Cao, X.; Jin, D.; Yang, B.; Guo, Y.
  • Graph adversarial self-supervised learning
    Yang, L.; Zhang, L.; Yang, W.
  • Unsupervised word sense disambiguation rivaling supervised methods
    Yarowsky, D.
  • Sparse graph attention networks
    Ye, Y.; Ji, S.
  • Identity-aware graph neural networks
    You, J.; Gomes-Selman, J. M.; Ying, R.; Leskovec, J.
  • Graph contrastive learning automated
    You, Y.; Chen, T.; Shen, Y.; Wang, Z.
  • Graph contrastive learning with augmentations
    You, Y.; Chen, T.; Sui, Y.; Chen, T.; Wang, Z.; Shen, Y.
  • When does self-supervision help graph convolutional networks? In ICML, 2020
    You, Y.; Chen, T.; Wang, Z.; Shen, Y.
  • Labelinvariant augmentation for semi-supervised graph classification
    Yue, H.; Zhang, C.; Zhang, C.; Liu, H.
  • Decoupling the depth and scope of graph neural networks
    Zeng, H.; Zhang, M.; Xia, Y.; Srivastava, A.; Malevich, A.; Kannan, R.; Prasanna, V.; Jin, L.; Chen, R.
  • Graphsaint: Graph sampling based inductive learning method
    Zeng, H.; Zhou, H.; Srivastava, A.; Kannan, R.; Prasanna, V.
  • Contrastive self-supervised learning for graph classification
    Zeng, J.; Xie, P.
  • Defensevgae: Defending against adversarial attacks on graph data via a variational graph autoencoder
    Zhang, A.; Ma, J.
  • Improving the training of graph neural networks with consistency regularization
    Zhang, C.; He, Y.; Cen, Y.; Hou, Z.; Tang, J.
  • mixup: Beyond empirical risk minimization
    Zhang, H.; Cisse, M.; Dauphin, Y. N.; Paz, D. Lopez-
  • Graph-bert: Only attention is needed for learning graph representations
    Zhang, J.; Zhang, H.; Xia, C.; Sun, L.
  • Nested graph neural networks
    Zhang, M.; Li, P.
  • Graph convolutional networks: a comprehensive review
    Zhang, S.; Tong, H.; Xu, J.; Maciejewski, R.
  • Gnnguard: Defending graph neural networks against adversarial attacks
    Zhang, X.; Zitnik, M.
  • Bayesian graph convolutional neural networks for semi-supervised classification
    Zhang, Y.; Pal, S.; Coates, M.; Ustebay, D.
  • Adaptive diffusion in graph neural networks
    Zhao, J.; Dong, Y.; Ding, M.; Kharlamov, E.; Tang, J.
  • Heterogeneous graph structure learning for graph neural networks
    Zhao, J.; Wang, X.; Shi, C.; Hu, B.; Song, G.; Ye, Y.
  • Learning from counterfactual links for link prediction
    Zhao, T.; Liu, G.; Wang, D.; Yu, W.; Jiang, M.
  • Data augmentation for graph neural networks
    Zhao, T.; Liu, Y.; Neves, L.; Woodford, O.; Jiang, M.; Shah, N.
  • Graphsmote: Imbalanced node classification on graphs with graph neural networks
    Zhao, T.; Zhang, X.; Wang, S.
  • Robust graph representation learning via neural sparsification
    Zheng, C.; Zong, B.; Cheng, W.; Song, D.; Ni, J.; Yu, W.; Chen, H.; Wang, W.
  • Deep graph contrastive representation learning
    Zhu, Y.; Xu, Y.; Yu, F.; Liu, Q.; Wu, S.; Wang, L.
  • Graph contrastive learning with adaptive augmentation
    Zhu, Y.; Xu, Y.; Yu, F.; Liu, Q.; Wu, S.; Wang, L.
  • Layer-dependent importance sampling for training deep and large graph convolutional networks
    Zou, D.; Hu, Z.; Wang, Y.; Jiang, S.; Sun, Y.; Gu, Q.