epfml/sparsifiedSGD epfml/sparsifiedSGD

  • Stars
    star
    54
  • Rank 544,902 (Top 11 %)
  • Language
    Jupyter Notebook
  • License
    Apache License 2.0
  • Created over 6 years ago
  • Updated about 6 years ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
epfml/sparsifiedSGD
github

epfml

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Sparsified SGD with Memory: https://arxiv.org/abs/1809.07599

More Repositories

1

ML_course

EPFL Machine Learning Course, Fall 2024
Jupyter Notebook
1,254
star
2

sent2vec

General purpose unsupervised sentence representations
C++
1,192
star
3

OptML_course

EPFL Course - Optimization for Machine Learning - CS-439
Jupyter Notebook
1,122
star
4

attention-cnn

Source code for "On the Relationship between Self-Attention and Convolutional Layers"
Python
1,073
star
5

landmark-attention

Landmark Attention: Random-Access Infinite Context Length for Transformers
Python
258
star
6

federated-learning-public-code

Python
157
star
7

disco

DISCO is a code-free and installation-free browser platform that allows any non-technical user to collaboratively train machine learning models without sharing any private data.
TypeScript
152
star
8

collaborative-attention

Code for Multi-Head Attention: Collaborate Instead of Concatenate
Python
148
star
9

powersgd

Practical low-rank gradient compression for distributed optimization: https://arxiv.org/abs/1905.13727
Python
137
star
10

dynamic-sparse-flash-attention

Jupyter Notebook
129
star
11

DenseFormer

Python
74
star
12

llm-baselines

Python
68
star
13

ChocoSGD

Decentralized SGD and Consensus with Communication Compression: https://arxiv.org/abs/1907.09356
Python
59
star
14

optML-pku

summer school materials
42
star
15

LocalSGD-Code

Python
42
star
16

error-feedback-SGD

SGD with compressed gradients and error-feedback: https://arxiv.org/abs/1901.09847
Jupyter Notebook
28
star
17

interpret-lm-knowledge

Extracting knowledge graphs from language models as a diagnostic benchmark of model performance (NeurIPS XAI 2021).
Jupyter Notebook
22
star
18

byzantine-robust-optimizer

Learning from history for Byzantine Robustness
Jupyter Notebook
21
star
19

Bi-Sent2Vec

Robust Cross-lingual Embeddings from Parallel Sentences
C++
20
star
20

opt-summerschool

Short Course on Optimization for Machine Learning - Slides and Practical Labs - DS3 Data Science Summer School, June 24 to 28, 2019, Paris, France
Jupyter Notebook
20
star
21

cola

CoLa - Decentralized Linear Learning: https://arxiv.org/abs/1808.04883
Python
18
star
22

opt-shortcourse

Short Course on Optimization for Machine Learning - Slides and Practical Lab - Pre-doc Summer School on Learning Systems, July 3 to 7, 2017, Zürich, Switzerland
Jupyter Notebook
18
star
23

powergossip

Code for "Practical Low-Rank Communication Compression in Decentralized Deep Learning"
Python
15
star
24

byzantine-robust-noniid-optimizer

Python
15
star
25

X2Static

X2Static embeddings
Python
12
star
26

kubernetes-setup

MLO group setup for kubernetes cluster
Dockerfile
12
star
27

topology-in-decentralized-learning

Code related to ’Beyond spectral gap: The role of the topology in decentralized learning‘.
Python
10
star
28

quasi-global-momentum

Python
10
star
29

relaysgd

Code for the paper “RelaySum for Decentralized Deep Learning on Heterogeneous Data”
Jupyter Notebook
10
star
30

piecewise-affine-multiplication

Python
7
star
31

rotational-optimizers

Python
6
star
32

byzantine-robust-decentralized-optimizer

Jupyter Notebook
6
star
33

uncertainity-estimation

Code for the paper “The Peril of Popular Deep Learning Uncertainty Estimation Methods”
Jupyter Notebook
6
star
34

getting-started

Python
6
star
35

text_to_image_generation

Python
5
star
36

easy-summary

difficulty-guided text summarization
Python
5
star
37

FeAI

Federated Learning with TensorFlow.js
Vue
4
star
38

ghost-noise

Python
3
star
39

autoTrain

Open Challenge - Automatic Training for Deep Learning
Python
3
star
40

pax

JAX-like API for PyTorch
Python
3
star
41

personalized-collaborative-llms

Python
2
star
42

phantomedicus

MedSurge: medical survey generator
Jupyter Notebook
1
star