diaoenmao/HeteroFL-Computation-and-Communication-Efficient-Federated-Learning-for-Heterogeneous-Clients

Stars
141
Rank 258,483 (Top 6 %)
Language
Python
License
MIT License
Created almost 5 years ago
Updated over 1 year ago

diaoenmao/HeteroFL-Computation-and-Communication-Efficient-Federated-Learning-for-Heterogeneous-Clients

diaoenmao

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

[ICLR 2021] HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients

HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients

[ICLR 2021] This is an implementation of HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients

Global model parametersWgare distributed to m=6 local clients with p=3 computation complexity levels.

Requirements

see requirements.txt

Instruction

Global hyperparameters are configured in config.yml
Hyperparameters can be found at process_control() in utils.py
fed.py contrains aggregation and separation of subnetworks

Examples

Train MNIST dataset (IID) with CNN model, 100 users, active rate 0.1, model split 'Fix', model split mode 'a-b (20%-80%)', BatchNorm, Scaler (True) , Masked CrossEntropy (True)
```
python train_classifier_fed.py --data_name MNIST --model_name conv --control_name 1_100_0.1_iid_fix_a2-b8_bn_1_1
```
Train CIFAR10 dataset (Non-IID 2 classes) with ResNet model, 10 users, active rate 0.1, model split 'Dynamic', model split mode 'a-b-c (uniform)', GroupNorm, Scaler (False) , Masked CrossEntropy (False)
```
python train_classifier_fed.py --data_name CIFAR10 --model_name resnet18 --control_name 1_10_0.1_non-iid-2_dynamic_a1-b1-c1_gn_0_0
```
Test WikiText2 dataset with Transformer model, 100 users, active rate 0.01, model split 'Fix', model split mode 'a (50%), b(50%)', No Normalization, Scaler (True) , Masked CrossEntropy (False)
```
python test_transformer_fed.py --data_name WikiText2 --model_name transformer --control_name 1_100_0.01_iid_fix_a5-b5_none_1_0
```

Results

Interpolation experimental results for MNIST (IID) dataset between global model complexity ((a) a, (b) b, (c) c, (d) d) and various smaller model complexities.

Interpolation experimental results for CIFAR10 (IID) dataset between global model complexity ((a) a, (b) b, (c) c, (d) d) and various smaller model complexities.

Interpolation experimental results for WikiText2 (IID) dataset between global model complexity ((a) a, (b) b, (c) c, (d) d) and various smaller model complexities.

Acknowledgement

Enmao Diao
Jie Ding
Vahid Tarokh

FPGA-CNN

FPGA implementation of Cellular Neural Network (CNN)

SemiFL-Semi-Supervised-Federated-Learning-for-Unlabeled-Clients-with-Alternate-Training

[NeurIPS 2022] SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Training

Library-Management-System

A Library Management System with PHP and MySQL

Pruning-Deep-Neural-Networks-from-a-Sparsity-Perspective

[ICLR 2023] Pruning Deep Neural Networks from a Sparsity Perspective

RPipe

Research Pipeline (RPipe)

Speech-Emotion-Recognition-with-Dual-Sequence-LSTM-Architecture

[ICASSP 2020] Speech Emotion Recognition with Dual-Sequence LSTM Architecture

Belief-Propagation

Implementation of Generalized Belief Propagation and Convergence Rate Analysis

MIREX-Audio-Melody-Extraction-Data-Analysis

Music Information Retrieval Evaluation eXchange (MIREX) Audio Melody Extraction Data Analysis during 2009-2014

Monophonic-Pitch-Tracking

Monophonic Ptich Tracking algorithms

GAL-Gradient-Assisted-Learning-for-Decentralized-Multi-Organization-Collaborations

[NeurIPS 2022] GAL: Gradient Assisted Learning for Decentralized Multi-Organization Collaborations

Dimension-Reduced-Turbulent-Flow-Data-From-Deep-Vector-Quantizers

[Journal of Turbulence, DCC 2022] Dimension Reduced Turbulent Flow Data From Deep Vector Quantizers

DRASIC-Distributed-Recurrent-Autoencoder-for-Scalable-Image-Compression

[DCC 2020] DRASIC: Distributed Recurrent Autoencoder for Scalable Image Compression

Deep-Voice-Conversion

Multimodal-Controller-for-Generative-Models

[CVMI 2022] Multimodal Controller for Generative Models

ColA-Collaborative-Adaptation-with-Gradient-Learning

[arXiv] ColA: Collaborative Adaptation with Gradient Learning

Computer-Simulation

Computer Simulation Projects originally from CX 4230

Jupyter Notebook

FLPipe

Federated Learning Pipeline

Deep-Audio-Signal-Coding

Jupyter Notebook

Personalized-Federated-Recommender-Systems-with-Private-and-Partially-Federated-AutoEncoders

[Asilomar 2022] Personalized Federated Recommender Systems with Private and Partially Federated AutoEncoders

LLM-for-Recommender-Systems

Digital-Signal-Processing-Applications

Digital Signal Processing Applications originally from ECE 4271

Earthquake-Forecasting

Semi-Supervised-Federated-Learing-for-Keyword-Spotting

[ICME 2023] Semi-Supervised Federated Learing for Keyword Spotting

STF-Ivy-Dream-Works-MBTI-Test

A website for testing academic abilities and psychological tendencies for STF Ivy Dream Works