epfml/powergossip

Stars
15
Rank 1,371,379 (Top 28 %)
Language
Python
License
MIT License
Created over 4 years ago
Updated over 4 years ago

epfml/powergossip

epfml

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Code for "Practical Low-Rank Communication Compression in Decentralized Deep Learning"

ML_course

EPFL Machine Learning Course, Fall 2024

Jupyter Notebook

sent2vec

General purpose unsupervised sentence representations

OptML_course

EPFL Course - Optimization for Machine Learning - CS-439

Jupyter Notebook

attention-cnn

Source code for "On the Relationship between Self-Attention and Convolutional Layers"

landmark-attention

Landmark Attention: Random-Access Infinite Context Length for Transformers

federated-learning-public-code

disco

DISCO is a code-free and installation-free browser platform that allows any non-technical user to collaboratively train machine learning models without sharing any private data.

collaborative-attention

Code for Multi-Head Attention: Collaborate Instead of Concatenate

powersgd

Practical low-rank gradient compression for distributed optimization: https://arxiv.org/abs/1905.13727

dynamic-sparse-flash-attention

Jupyter Notebook

DenseFormer

llm-baselines

ChocoSGD

Decentralized SGD and Consensus with Communication Compression: https://arxiv.org/abs/1907.09356

sparsifiedSGD

Sparsified SGD with Memory: https://arxiv.org/abs/1809.07599

Jupyter Notebook

optML-pku

summer school materials

LocalSGD-Code

error-feedback-SGD

SGD with compressed gradients and error-feedback: https://arxiv.org/abs/1901.09847

Jupyter Notebook

interpret-lm-knowledge

Extracting knowledge graphs from language models as a diagnostic benchmark of model performance (NeurIPS XAI 2021).

Jupyter Notebook

byzantine-robust-optimizer

Learning from history for Byzantine Robustness

Jupyter Notebook

Bi-Sent2Vec

Robust Cross-lingual Embeddings from Parallel Sentences

opt-summerschool

Short Course on Optimization for Machine Learning - Slides and Practical Labs - DS3 Data Science Summer School, June 24 to 28, 2019, Paris, France

Jupyter Notebook

cola

CoLa - Decentralized Linear Learning: https://arxiv.org/abs/1808.04883

opt-shortcourse

Short Course on Optimization for Machine Learning - Slides and Practical Lab - Pre-doc Summer School on Learning Systems, July 3 to 7, 2017, Zürich, Switzerland

Jupyter Notebook

byzantine-robust-noniid-optimizer

X2Static

X2Static embeddings

kubernetes-setup

MLO group setup for kubernetes cluster

topology-in-decentralized-learning

Code related to ’Beyond spectral gap: The role of the topology in decentralized learning‘.

quasi-global-momentum

relaysgd

Code for the paper “RelaySum for Decentralized Deep Learning on Heterogeneous Data”

Jupyter Notebook

piecewise-affine-multiplication

rotational-optimizers

byzantine-robust-decentralized-optimizer

Jupyter Notebook

uncertainity-estimation

Code for the paper “The Peril of Popular Deep Learning Uncertainty Estimation Methods”

Jupyter Notebook

getting-started

text_to_image_generation

easy-summary

difficulty-guided text summarization

FeAI

Federated Learning with TensorFlow.js

ghost-noise

autoTrain

Open Challenge - Automatic Training for Deep Learning

pax

JAX-like API for PyTorch

personalized-collaborative-llms

phantomedicus

MedSurge: medical survey generator

Jupyter Notebook