Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Federated Graph Machine Learning

Federated Graph Machine Learning Graph machine learning has gained great attention in both academia and industry recently. Most of the graph machine learning models, such as Graph Neural Networks (GNNs), are trained over massive graph data. However, in many realworld scenarios, such as hospitalization prediction in healthcare systems, the graph data is usually stored at multiple data owners and cannot be directly accessed by any other parties due to privacy concerns and regulation restrictions. Federated Graph Machine Learning (FGML) is a promising solution to tackle this challenge by training graph machine learning models in a federated manner. In this survey, we conduct a comprehensive review of the literature in FGML. Specifically, we first provide a new taxonomy to divide the existing problems in FGML into two settings, namely, FL with structured data and structured FL. Then, we review the mainstream techniques in each setting and elaborate on how they address the challenges under FGML. In addition, we summarize the real-world applications of FGML from different domains and introduce open graph datasets and platforms adopted in FGML. Finally, we present several limitations in the existing studies with promising research directions in this field. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM SIGKDD Explorations Newsletter Association for Computing Machinery

Federated Graph Machine Learning

Download PDF
16 pages

Loading next page...
 
/lp/association-for-computing-machinery/federated-graph-machine-learning-idO5duG3zm
Publisher
Association for Computing Machinery
Copyright
Copyright © 2022 Copyright is held by the owner/author(s)
ISSN
1931-0145
eISSN
1931-0153
DOI
10.1145/3575637.3575644
Publisher site
See Article on Publisher Site

Abstract

Graph machine learning has gained great attention in both academia and industry recently. Most of the graph machine learning models, such as Graph Neural Networks (GNNs), are trained over massive graph data. However, in many realworld scenarios, such as hospitalization prediction in healthcare systems, the graph data is usually stored at multiple data owners and cannot be directly accessed by any other parties due to privacy concerns and regulation restrictions. Federated Graph Machine Learning (FGML) is a promising solution to tackle this challenge by training graph machine learning models in a federated manner. In this survey, we conduct a comprehensive review of the literature in FGML. Specifically, we first provide a new taxonomy to divide the existing problems in FGML into two settings, namely, FL with structured data and structured FL. Then, we review the mainstream techniques in each setting and elaborate on how they address the challenges under FGML. In addition, we summarize the real-world applications of FGML from different domains and introduce open graph datasets and platforms adopted in FGML. Finally, we present several limitations in the existing studies with promising research directions in this field.

Journal

ACM SIGKDD Explorations Newsletter – Association for Computing Machinery

Published: Dec 5, 2022

References

  • A survey on homomorphic encryption schemes: Theory and implementation
    Acar, A.; Aksu, H.; Uluagac, A. S.; Conti, M.
  • A federated multigraph integration approach for connectional brain template learning
    Bayram, H. C.; Rekik, I.
  • Towards federated learning at scale: System design
    Bonawitz, K.; Eichner, H.; Grieskamp, W.; Huba, D.; Ingerman, A.; Ivanov, V.; Kiddon, C.; Konecn'y, J.; Mazzocchi, S.; McMahan, B.
  • Translating embeddings for modeling multi-relational data
    Bordes, A.; Usunier, N.; Garcia-Duran, A.; Weston, J.; Yakhnenko, O.
  • Federated learning of predictive models from federated electronic health records
    Brisimi, T. S.; Chen, R.; Mela, T.; Olshevsky, A.; Paschalidis, I. C.; Shi, W.
  • Line graph neural networks for link prediction
    Cai, L.; Li, J.; Wang, J.; Ji, S.
  • Cluster-driven graph federated learning over multiple domains
    Caldarola, D.; Mancini, M.; Galasso, F.; Ciccone, M.; Rodola, E.; Caputo, B.
  • Leaf: A benchmark for federated settings
    Caldas, S.; Duddu, S. M. K.; Wu, P.; Li, T.; Konecn'y, J.; McMahan, H. B.; Smith, V.; Talwalkar, A.
  • Federated learning for site aware chest radiograph screening
    Chakravarty, A.; Kar, A.; Sethuraman, R.; Sheet, D.
  • Fedgl: federated graph learning framework with global selfsupervision
    Chen, C.; Hu, W.; Xu, Z.; Zheng, Z.
  • Fedgraph: Federated graph learning with intelligent sampling
    Chen, F.; Li, P.; Miyazaki, T.; Wu, C.
  • Personalized federated learning with graph
    Chen, F.; Long, G.; Wu, Z.; Zhou, T.; Jiang, J.
  • Structured graph convolutional networks with stochastic masks for recommender systems
    Chen, H.; Wang, L.; Lin, Y.; Yeh, C.-C. M.; Wang, F.; Yang, H.
  • Graph-fraudster: Adversarial attacks on graph neural network-based vertical federated learning
    Chen, J.; Huang, G.; Zheng, H.; Yu, S.; Jiang, W.; Cui, C.
  • Fede: Embedding knowledge graphs in federated setting
    Chen, M.; Zhang, W.; Yuan, Z.; Jia, Y.; Chen, H.
  • A simple framework for contrastive learning of visual representations
    Chen, T.; Kornblith, S.; Norouzi, M.; Hinton, G.
  • Fedsgc: Federated simple graph convolution for node classification
    Cheung, T.-H.; Dai, W.; Li, S.
  • Learning phrase representations using rnn encoder-decoder for statistical machine translation
    Cho, K.; van Merrienboer, B.; Gucehre, C.; Bahdanau, D.; Bougares, F.; Schwenk, H.; Bengio, Y.
  • Addressing algorithmic disparity and performance inconsistency in federated learning
    Cui, S.; Pan, W.; Liang, J.; Zhang, C.; Wang, F.
  • Symbolic analysis meets federated learning to enhance malware identifier
    Dam, K. H. T.; Van Ouytsel, C.-H. B.; Legay, A.
  • Inductive entity representations from text via link prediction
    Daza, D.; Cochez, M.; Groth, P.
  • Convolutional 2d knowledge graph embeddings
    Dettmers, T.; Minervini, P.; Stenetorp, P.; Riedel, S.
  • The autism brain imaging data exchange: Towards a large-scale evaluation of the intrinsic brain architecture in autism
    Di Martino, A.; Yan, C.-G.; Li, Q.; Denio, E.; Castellanos, F. X.; Alaerts, K.; Anderson, J. S.; Assaf, M.; Bookheimer, S. Y.; Dapretto, M.
  • A new look and convergence rate of federated multi-task learning with laplacian regularization
    Dinh, C. T.; Vu, T. T.; Tran, N. H.; Dao, M. N.; Zhang, H.
  • The yahoo! music dataset and kdd-cup'11
    Dror, G.; Koenigstein, N.; Koren, Y.; Weimer, M.
  • Fairfed: Enabling group fairness in federated learning
    Ezzeldin, Y. H.; Yan, S.; He, C.; Ferrara, E.; Avestimehr, S.
  • Fast graph representation learning with pytorch geometric
    Fey, M.; Lenssen, J. E.
  • Model-agnostic meta-learning for fast adaptation of deep networks
    Finn, C.; Abbeel, P.; Levine, S.
  • Graph neural networks for recommender system
    Gao, C.; Wang, X.; He, X.; Li, Y.
  • Differentially private federated learning: A client level perspective
    Geyer, R. C.; Klein, T.; Nabi, M.
  • An efficient framework for clustered federated learning
    Ghosh, A.; Chung, J.; Yin, D.; Ramchandran, K.
  • Citeseer: An automatic citation indexing system
    Giles, C. L.; Bollacker, K. D.; Lawrence, S.
  • Federated graph neural network for cross-graph node classification
    Guan, Z.; Li, Y.; Xue, Z.; Liu, Y.; Gao, H.; Shao, Y.
  • A novel bayesian similarity measure for recommender systems
    Guo, G.; Zhang, J.; Yorke-Smith, N.
  • Deep graph normalizer: A geometric deep learning approach for estimating connectional brain templates
    Gurbuz, M. B.; Rekik, I.
  • Locationtrails: A federated approach to learning location embeddings
    Gurukar, S.; Parthasarathy, S.; Ramnath, R.; Calder, C.; Moosavi, S.
  • Inductive representation learning on large graphs
    Hamilton, W.; Ying, Z.; Leskovec, J.
  • A collective learning framework to boost gnn expressiveness for node classification
    Hang, M.; Neville, J.; Ribeiro, B.
  • Fedgraphnn: A federated learning system and benchmark for graph neural networks
    He, C.; Balasubramanian, K.; Ceyani, E.; Yang, C.; Xie, H.; Sun, L.; He, L.; Yang, L.; Yu, P. S.; Rong, Y.
  • Spreadgnn: Serverless multi-task federated learning for graph neural networks
    He, C.; Ceyani, E.; Balasubramanian, K.; Annavaram, M.; Avestimehr, S.
  • Fedml: A research library and benchmark for federated machine learning
    He, C.; Li, S.; So, J.; Zeng, X.; Zhang, M.; Wang, H.; Wang, X.; Vepakomma, P.; Singh, A.; Qiu, H.
  • Deep residual learning for image recognition
    He, K.; Zhang, X.; Ren, S.; Sun, J.
  • Fedgcn: Federated learning-based graph convolutional networks for non-euclidean spatial data
    Hu, K.; Wu, J.; Li, Y.; Lu, M.; Weng, L.; Xia, M.
  • Open graph benchmark: Datasets for machine learning on graphs
    Hu, W.; Fey, M.; Zitnik, M.; Dong, Y.; Ren, H.; Liu, B.; Catasta, M.; Leskovec, J.
  • Densely connected convolutional networks
    Huang, G.; Liu, Z.; Van Der Maaten, L.; Weinberger, K. Q.
  • Personalized cross-silo federated learning on non-iid data
    Huang, Y.; Chu, L.; Zhou, Z.; Wang, L.; Liu, J.; Pei, J.; Zhang, Y.
  • Big data and its technical challenges
    Jagadish, H. V.; Gehrke, J.; Labrinidis, A.; Papakonstantinou, Y.; Patel, J. M.; Ramakrishnan, R.; Shahabi, C.
  • A matrix factorization technique with trust propagation for recommendation in social networks
    Jamali, M.; Ester, M.
  • Federated dynamic graph neural networks with secure aggregation for video-based distributed surveillance
    Jiang, M.; Jung, T.; Karl, R.; Zhao, T.
  • Graph structure learning for robust graph neural networks
    Jin, W.; Ma, Y.; Liu, X.; Tang, X.; S.Wang, ; Tang, J.
  • Learning graph structure with a finite-state automaton layer
    Johnson, D.; Larochelle, H.; Tarlow, D.
  • Pategan: Generating synthetic data with differential privacy guarantees
    Jordon, J.; Yoon, J.; Van Der Schaar, M.
  • Advances and open problems in federated learning
    Kairouz, P.; McMahan, H. B.; Avent, B.; Bellet, A.; Bennis, M.; Bhagoji, A. N.; Bonawitz, K.; Charles, Z.; Cormode, G.; Cummings, R.
  • Semi-supervised classification with graph convolutional networks
    Kipf, T. N.; Welling, M.
  • Peer-to-peer federated learning on graphs
    Lalitha, A.; Kilinc, O. C.; Javidi, T.; Koushanfar, F.
  • Spatial-temporal fusion graph neural networks for traffic flow forecasting
    Li, M.; Zhu, Z.
  • Ditto: Fair and robust federated learning through personalization
    Li, T.; Hu, S.; Beirami, A.; Smith, V.
  • Federated learning: Challenges, methods, and future directions
    Li, T.; Sahu, A. K.; Talwalkar, A.; Smith, V.
  • Federated meta-learning for spatial-temporal prediction
    Li, W.; Wang, S.
  • A new directed divergence measure and its characterization
    Lin, J.; Wong, S.
  • Improving federated relational data modeling via basis alignment and weight penalty
    Lin, Y.; Chen, C.; Chen, C.; Wang, L.
  • Deep gradient compression: Reducing the communication bandwidth for distributed training
    Lin, Y.; Han, S.; Mao, H.; Wang, Y.; Dally, W. J.
  • Federated graph neural networks: Overview, techniques and challenges
    Liu, R.; Yu, H.
  • Glint: Decentralized federated graph learning with traffic throttling and flow scheduling
    Liu, T.; Li, P.; Gu, Y.
  • Towards unsupervised deep graph structure learning
    Liu, Y.; Zheng, Y.; Zhang, D.; Chen, H.; Peng, H.; Pan, S.
  • Federated social recommendation with graph neural network
    Liu, Z.; Yang, L.; Fan, Z.; Peng, H.; Yu, P. S.
  • Federated approach for privacy-preserving traffic prediction using graph convolutional network
    Lonare, S.; Bhramaramba, R.
  • Stfl: A temporal-spatial federated learning framework for graph neural networks
    Lou, G.; Liu, Y.; Zhang, T.; Zheng, X.
  • Decentralized federated learning for electronic health records
    Lu, S.; Zhang, Y.; Wang, Y.
  • Learn electronic health records by fully decentralized federated learning
    Lu, S.; Zhang, Y.; Wang, Y.; Mack, C.
  • Recommender systems with social regularization
    Ma, H.; Zhou, D.; Liu, C.; Lyu, M. R.; King, I.
  • Masked graph modeling for molecule generation
    Mahmood, O.; Mansimov, E.; Bonneau, R.; Cho, K.
  • Packet routing with graph attention multi-agent reinforcement learning
    Mai, X.; Fu, Q.; Chen, Y.
  • Fl-disco: Federated generative adversarial network for graph-based molecule drug discovery
    Manu, D.; Sheng, Y.; Yang, J.; Deng, J.; Geng, T.; Li, A.; Ding, C.; Jiang, W.; Yang, L.
  • Automating the construction of internet portals with machine learning
    McCallum, A. K.; Nigam, K.; Rennie, J.; Seymore, K.
  • Communication-efficient learning of deep networks from decentralized data
    McMahan, B.; Moore, E.; Ramage, D.; Hampson, S.; Arcas, B. A. y
  • Sgnn: A graph neural network based federated learning approach by hiding structure
    Mei, G.; Guo, Z.; Liu, S.; Pan, L.
  • Cross-node federated graph neural network for spatio-temporal data modeling
    Meng, C.; Rambhatla, S.; Liu, Y.
  • Movielens unplugged: Experiences with an occasionally connected recommender system
    Miller, B. N.; Albert, I.; Lam, S. K.; Konstan, J. A.; Riedl, J.
  • Tudataset: A collection of benchmark datasets for learning with graphs
    Morris, C.; Kriege, N. M.; Bause, F.; Kersting, K.; Mutzel, P.; Neumann, M.
  • A vertical federated learning framework for graph convolutional network
    Ni, X.; Xu, X.; Lyu, L.; Meng, C.; Wang, W.
  • California department of transportation
    Transportation, C. D. of
  • Simultaneous inference for misaligned multivariate functional data
    Olsen, N. L.; Markussen, B.; Raket, L. L.
  • Transfer graph neural networks for pandemic forecasting
    Panagopoulos, G.; Nikolentzos, G.; Vazirgiannis, M.
  • Towards deep federated defenses against malware in cloud ecosystems
    Payne, J.; Kundu, A.
  • Decentralized federated graph neural networks
    Pei, Y.; Mao, R.; Liu, Y.; Chen, C.; Xu, S.; Qiang, F.; Tech, B. E.
  • Differentially private federated knowledge graphs embedding
    Peng, H.; Li, H.; Song, Y.; Zheng, V.; Li, J.
  • Fedni: Federated graph learning with network inpainting for population-based disease prediction
    Peng, L.; Wang, N.; Dvornek, N.; Zhu, X.; Li, X.
  • Alzheimer's disease neuroimaging initiative (adni): clinical characterization
    Petersen, R. C.; Aisen, P.; Beckett, L. A.; Donohue, M.; Gamst, A.; Harvey, D. J.; Jack, C.; Jagust, W.; Shaw, L.; Toga, A.
  • struc2vec: Learning node representations from structural identity
    Ribeiro, L. F.; Saverese, P. H.; Figueiredo, D. R.
  • A graph federated architecture with privacy preserving learning
    Rizk, E.; Sayed, A. H.
  • Graph neural networks for privacy-preserving recommendation with secure hardware
    Ru, S.; Zhang, B.; Jie, Y.; Zhang, C.; Wei, L.; Gu, C.
  • Grafehty: Graph neural network using federated learning for human activity recognition
    Sarkar, A.; Sen, T.; Roy, A. K.
  • Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints
    Sattler, F.; M¨uller, K.-R.; Samek, W.
  • d
    Schlichtkrull, M.; Kipf, T. N.; Bloem, P.; v, R.
  • Collective classification in network data
    Sen, P.; Namata, G.; Bilgic, M.; Getoor, L.; Galligher, B.; Eliassi-Rad, T.
  • Personalized federated learning using hypernetworks
    Shamsian, A.; Navon, A.; Fetaya, E.; Chechik, G.
  • Pitfalls of graph neural network evaluation
    Shchur, O.; Mumme, M.; Bojchevski, A.; Gunnemann, S.
  • Towards federated graph learning for collaborative financial crimes detection
    Suzumura, T.; Zhou, Y.; Baracaldo, N.; Ye, G.; Houck, K.; Kawahara, R.; Anwar, A.; Stavarache, L. L.; Watanabe, Y.; Loyola, P.
  • Towards personalized federated learning
    Tan, A. Z.; Yu, H.; Cui, L.; Yang, Q.
  • mtrust: Discerning multifaceted trust in a connected world
    Tang, J.; Gao, H.; Liu, H.
  • Arnetminer: Extraction and mining of academic social networks
    Tang, J.; Zhang, J.; Yao, L.; Li, J.; Zhang, L.; Su, Z.
  • Federated learning of user mobility anomaly based on graph attention networks
    Tang, Y.; Jia, R.; Zhou, X.; Li, Z.; Jin, H.; Zhao, C.
  • Dynamic neural graphs based federated reptile for semisupervised multi-tasking in healthcare applications
    Thakur, A.; Sharma, P.; Clifton, D. A.
  • Graph attention networks
    Velickovi´c, P.; Cucurull, G.; Casanova, A.; Romero, A.; Lio, P.; Bengio, Y.
  • Optimal transport: old and new
    Villani, C.
  • The eu general data protection regulation (gdpr)
    Voigt, P.; Von dem Bussche, A.
  • Graphfl: A federated learning framework for semi-supervised node classification on graphs
    Wang, B.; Li, A.; Li, H.; Chen, Y.
  • Flagcns: Federated learning framework for automatic graph convolutional network search
    Wang, C.; Chen, B.; Li, G.; Wang, H.
  • Relational message passing for knowledge graph completion
    Wang, H.; Ren, H.; Leskovec, J.
  • Federatedscope-gnn: Towards a unified, comprehensive and efficient package for federated graph learning
    Wang, Z.; Kuang, W.; Xie, Y.; Yao, L.; Li, Y.; Ding, B.; Zhou, J.
  • Online disease diagnosis with inductive heterogeneous graph convolutional networks
    Wang, Z.; Wen, R.; Chen, X.; Cao, S.; Huang, S.-L.; Qian, B.; Zheng, Y.
  • Fedgnn: Federated graph neural network for privacy-preserving recommendation
    C.Wu, ; F.Wu, ; Cao, Y.; Huang, Y.; Xie, X.
  • A federated graph neural network framework for privacypreserving personalization
    C.Wu, ; F.Wu, ; Lyu, L.; Qi, T.; Huang, Y.; Xie, X.
  • Moleculenet: a benchmark for molecular machine learning
    Wu, Z.; Ramsundar, B.; Feinberg, E. N.; Gomes, J.; Geniesse, C.; Pappu, A. S.; Leswing, K.; Pande, V.
  • Multi-level federated graph learning and self-attention based personalized wi-fi indoor fingerprint localization
    Wu, Z.; Wu, X.; Long, Y.
  • Federated graph classification over non-iid graphs
    Xie, H.; Ma, J.; Xiong, L.; Yang, C.
  • Federatedscope: A comprehensive and flexible federated learning platform via message passing
    Xie, Y.; Wang, Z.; Chen, D.; Gao, D.; Yao, L.; Kuang, W.; Li, Y.; Ding, B.; Zhou, J.
  • Decentralized federated learning via sgd over wireless d2d networks
    Xing, H.; Simeone, O.; Bi, S.
  • Big-fed: Bilevel optimization enhanced graph-aided federated learning
    Xing, P.; Lu, S.; Wu, L.; Yu, H.
  • More is better (mostly): On the backdoor attacks in federated graph neural networks
    Xu, J.; R.Wang, ; Liang, K.; Picek, S.
  • How powerful are graph neural networks? In ICLR, 2019
    Xu, K.; Hu, W.; Leskovec, J.; Jegelka, S.
  • Federated machine learning: Concept and applications
    Yang, Q.; Liu, Y.; Chen, T.; Tong, Y.
  • Federated traffic engineering with supervised learning in multiregion networks
    Ye, M.; Zhang, J.; Guo, Z.; Chao, H. J.
  • Fedstn: Graph representation driven federated learning for edge computing enabled urban traffic flow prediction
    Yuan, X.; Chen, J.; Yang, J.; Zhang, N.; Yang, T.; Han, T.; Taherkordi, A.
  • Multi-task federated learning for traffic prediction and its application to route planning
    Zeng, T.; Guo, J.; Kim, K. J.; Parsons, K.; Orlik, P.; Di Cairano, S.; Saad, W.
  • Ppsgcn: A privacy-preserving subgraph sampling based distributed gcn training method
    Zhang, B.; Luo, M.; Feng, S.; Liu, Z.; Zhou, J.; Zheng, Q.
  • A communication-efficient federated learning scheme for iot-based traffic forecasting
    Zhang, C.; Cui, L.; Yu, S.; James, J.
  • Fastgnn: A topological information protected federated learning approach for traffic speed forecasting
    Zhang, C.; Zhang, S.; James, J.; Yu, S.
  • Graph-based traffic forecasting via communication-efficient federated learning
    Zhang, C.; Zhang, S.; Yu, S.; James, J.
  • Federated graph learning--a position paper
    Zhang, H.; Shen, T.; Wu, F.; Yin, M.; Yang, H.; Wu, C.
  • Efficient federated learning on knowledge graphs via privacy-preserving relation embedding aggregation
    Zhang, K.; Wang, Y.; Wang, H.; Huang, L.; Yang, C.; Sun, L.
  • Subgraph federated learning with missing neighbor generation
    Zhang, K.; Yang, C.; Li, X.; Sun, L.; Yiu, S. M.
  • Traffic flow forecasting with spatial-temporal graph diffusion network
    Zhang, X.; Huang, C.; Xu, Y.; Xia, L.; Dai, P.; Bo, L.; Zhang, J.; Zheng, Y.
  • Motif-based graph self-supervised learning for molecular property prediction
    Zhang, Z.; Liu, Q.; Wang, H.; Lu, C.; Lee, C.-K.
  • Data augmentation for graph neural networks
    Zhao, T.; Liu, Y.; Neves, L.; Woodford, O.; Jiang, M.; Shah, N.
  • Graphsmote: Imbalanced node classification on graphs with graph neural networks
    Zhao, T.; Zhang, X.; Wang, S.
  • Asfgnn: Automated separated-federated graph neural network
    Zheng, L.; Zhou, J.; Chen, C.; Wu, B.; Wang, L.; Zhang, B.
  • Urban computing: Concepts, methodologies, and applications
    Zheng, Y.; Capra, L.; Wolfson, O.; Yang, H.
  • Vertically federated graph neural network for privacy-preserving node classification
    Zhou, J.; Chen, C.; Zheng, L.; Wu, H.; Wu, J.; Zheng, X.; Wu, B.; Liu, Z.; Wang, L.
  • Federated learning on non-iid data: A survey
    Zhu, H.; Xu, J.; Liu, S.; Jin, Y.
  • Federated learning of molecular properties in a heterogeneous setting
    Zhu, W.; White, A.; Luo, J.
  • Pysyft: A library for easy federated learning
    Ziller, A.; Trask, A.; Lopardo, A.; Szymkow, B.; Wagner, B.; Bluemke, E.; Nounahon, J.-M.; Palmbach, J. Passerat-; Prakash, K.; Rose, N.