Discover @Shigangli Open Source projects

Shigang Li (@Shigangli)

Shigangli

Stars
178
Global Rank 134,233 (Top 5 %)
Followers 54
Following 4
Registered over 9 years ago
Most used languages

Python
40.0 %

C++
30.0 %

C
20.0 %

Cuda 10.0 %
Location 🇨🇳 China
Country Total Rank 20,595
Country Ranking

C++
1,580

Python
5,411

C
6,976

Magicube

Magicube is a high-performance library for quantized sparse matrix operations (SpMM and SDDMM) of deep learning on Tensor Cores.

Chimera

Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines.

Ok-Topk

Ok-Topk is a scheme for distributed training with sparse gradients. Ok-Topk integrates a novel sparse allreduce algorithm (less than 6k communication volume which is asymptotically optimal) with the decentralized parallel Stochastic Gradient Descent (SGD) optimizer, and its convergence is proved theoretically and empirically.

eager-SGD

Eager-SGD is a decentralized asynchronous SGD. It utilizes novel partial collectives operations to accumulate the gradients across all the processes.

SpMV-on-Many-Core

A cross-platform Sparse Matrix Vector Multiplication (SpMV) framework for many-core architectures (GPUs and Xeon Phi).

WAGMA-SGD

WAGMA-SGD is a decentralized asynchronous SGD based on wait-avoiding group model averaging. The synchronization is relaxed by making the collectives externally-triggerable, namely, a collective can be initiated without requiring that all the processes enter it. It partially reduces the data within non-overlapping groups of process, improving the parallel scalability.

COMPI

Cache-oblivious MPI all-to-all communications based on Morton order

DNN-cpp-proxies

C++/MPI proxies for distributed training of deep neural networks.

cnnARM