-
Stars4
-
Rank 3,286,764 (Top 66 %)
-
LanguagePython
-
License
Apache License 2.0 -
Created over 5 years ago
-
Updated over 5 years ago
Reviews
There are no reviews yet. Be the first to send feedback to the community and the maintainers!
Repository Details
An improved activation function for deep learning - Threshold Relu, or TRelu
More Repositories
1
Ranger-Deep-Learning-Optimizer
Ranger - a synergistic optimizer using RAdam (Rectified Adam), Gradient Centralization and LookAhead in one codebasePython
1,154
2
Ranger21
Ranger deep learning optimizer rewrite to use newest componentsPython
320
3
Best-Deep-Learning-Optimizers
Collection of the latest, greatest, deep learning optimizers (for Pytorch) - CNN, NLP suitableJupyter Notebook
201
4
mish
Mish Deep Learning Activation Function for PyTorch / FastAIJupyter Notebook
161
5
res2net-plus
Res2Net architecture with improved stem and Mish activation functionPython
136
6
Ranger-Mish-ImageWoof-5
Repo to build on / reproduce the record breaking Ranger-Mish-SelfAttention setup on FastAI ImageWoof dataset 5 epochsJupyter Notebook
116
7
training-detr
Unofficial Colab on how to train DETR, the intelligent object detector, with your own dataset. DETR = Detection TransformerJupyter Notebook
40
8
transformer_central
Various transformers for FSDP researchJupyter Notebook
30
9
Ranger22
Testing various improvements to Ranger21 for 2022Python
18
10
mrnet-fastai
Deep Learning CNN using FastAI for the Stanford MRNet Knee MRI diagnosis challengeJupyter Notebook
16
11
FAdam_PyTorch
an implementation of FAdam (Fisher Adam) in PyTorchPython
16
12
Thunder-Detr
(unofficial) - customized fork of DETR, optimized for intelligent obj detection on 'real world' custom datasetsJupyter Notebook
12
13
triton_kernels_for_fun_and_profit
Custom kernels in Triton language for accelerating LLMsPython
8
14
fsdp_llm
FSDP optimizations for LLM trainingPython
6
15
t5_11
housing our model example of fine tuning an 11B t5 with FSDPPython
6
16
transformer_framework
framework for plug and play of various transformers (vision and nlp) with FSDPPython
6
17
FTSwishPlus
FTSwish with mean shifting added to increase performancePython
6
18
LightRelu
Customized PyTorch implementation of LiSHT (linear scaled hyperbolic tangent) activation function for deep learningPython
5
19
hyper_efficient_optimizers
Development of hyper efficient optimizers that can match/exceed AdamW, while using reduced memoryPython
5
20
fsdp_review
Some eval and profile routines for fsdp
4
21
auto-adaptive-ai
auto adaptive framework for intrinsic hyperparameter selection, adaptive padding, normalized weightsJupyter Notebook
4
22
sigma_reparam
Sigma Reparam for Transformers (based on Apple's paper)Python
3
23
EfficientNet-PyTorch
Unofficial port of Google's new EfficientNet to Pytorch and FastAIJupyter Notebook
3
24
RangerQH-Testing
Repo for running RangerQH + Res2NetPLus with LIP PoolingJupyter Notebook
3
25
facial-keypoint-detection
Facial keypoint detection CNN - custom architecture using partial convolution paddingJupyter Notebook
3
26
AutoOpt-for-FastAI
Integrate Ebay's AutoOpt Deep Learning Optimizer into the FastAI framework
3
27
skycraft2
Minecraft in the sky, written in PythonPython
2
28
perception_tools
additional utils for working with Unity perception packageJupyter Notebook
2
29
QuantFour_AdamW_Cuda
Fused 4bit AdamW in CudaPython
2
30
PolarBearLLM
testing new TransFormer, MoE, and TransNormer featuresPython
2
31
unet-seg
Jupyter Notebook
2
32
FTSwish
Flattened Threshold Swish Activation function - PyTorch implementationPython
2
33
coordinate_clipped_Optimizers
coordinate wise clipped Optimizers in PyTorchPython
2
34
snowfall
helpful image handling utils - abstracts various file and opencv and pil features into result oriented functionsPython
2
35
style-transfer-vgg
Artistic Style transfer using VGG19Jupyter Notebook
2
36
cuda-kernel-dev
in progress cuda kernels
Cuda
2
37
Curriculum-Learning-Dropout
Implementation of Curriculum Learning Dropout for FastAI frameworkJupyter Notebook
2
38
medExam
Training an AI with FSDP to take the US medical exam
1
39
5D-Compiler
Auto-Parallelization Compiler using 4D Parallel + Checkpointing (5D)Python
1
40
aot_fsdp
When AOT Autograd meets FSDP = large models train faster
1
41
alibi_positional_embeddings
Alibi in PyTorchPython
1
42
optimal-lr-finder
Automated optimal learning rate finder for PyTorch deep learning with FastAI
1
43
ft_linen
experiments with flax re-design to interop with pytorchPython
1
44
linear-graph-slam
Linear Graph SLAMJupyter Notebook
1
45
bfloat_optimizer
Pure bfloat AdamW+ tweaksPython
1
46
snake-id
FastAI deep learning classifier for snakes
1
47
Thunder
AI framework for flexible training and results review (pytorch, vision and tabular)
1
48
t5_finetuning
T5 and ExT5 fine tuningJupyter Notebook
1
49
pretrainer
FSDP codebase for pretraining large language models (LLM)Python
1
50
Fusion
Advanced yet low code framework for fully sharded distributed trainingPython
1
51
hsdp_demo
Tutorial repo for PyTorch FSDP running HSDP on single node.Python
1
52
image-captioning-cnn-lstm
Image captioning system combining CNN + LSTM for caption generationJupyter Notebook
1
53
self-tuning-ai
implementation of self tuning networks in pytorch, based on https://arxiv.org/pdf/1903.03088v1.pdf
1
54
triton_flashv2_alibi
working repo for Triton based Flash2 supporting alibi pos embeddingsPython
1
55
Pytorch_train_test_split
Function to randomize and split training data into train/test, from same directoryPython
1