ChristophReich1996/Scaling_Vision_Transformers_22B_Param ChristophReich1996/Scaling_Vision_Transformers_22B_Param

  • Stars
    star
    3
  • Rank 3,963,521 (Top 79 %)
  • Language
  • Created almost 2 years ago
  • Updated almost 2 years ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
ChristophReich1996/Scaling_Vision_Transformers_22B_Param
github

ChristophReich1996

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Reimplementation of the paper "Scaling Vision Transformers to 22 Billion Parameters" by Dehghani et al. [arXiv, 2023]

More Repositories

1

Swin-Transformer-V2

PyTorch reimplementation of the paper "Swin Transformer V2: Scaling Up Capacity and Resolution" [CVPR 2022].
Python
141
star
2

MaxViT

PyTorch reimplementation of the paper "MaxViT: Multi-Axis Vision Transformer" [arXiv 2022].
Python
135
star
3

Involution

PyTorch reimplementation of the paper "Involution: Inverting the Inherence of Convolution for Visual Recognition" (2D and 3D Involution) [CVPR 2021].
Python
101
star
4

Cell-DETR

Official and maintained implementation of the paper "Attention-Based Transformers for Instance Segmentation of Cells in Microstructures" [BIBM 2020].
Python
88
star
5

Semantic_Pyramid_for_Image_Generation

PyTorch reimplementation of the paper: "Semantic Pyramid for Image Generation" [CVPR 2020].
Python
46
star
6

Mode_Collapse

Mode collapse example of GANs in 2D (PyTorch).
Python
30
star
7

ECG_Classification

Official and maintained implementation of the paper "Exploring Novel Algorithms for Atrial Fibrillation Detection by Driving Graduate Level Education in Medical Machine Learning" (ECG-DualNet) [Physiological Measurement 2022].
Python
27
star
8

OSS-Net

Official and maintained implementation of the paper "OSS-Net: Memory Efficient High Resolution Semantic Segmentation of 3D Medical Data" [BMVC 2021].
Python
24
star
9

SmeLU

PyTorch reimplementation of the Smooth ReLU activation function proposed in the paper "Real World Large Scale Recommendation Systems Reproducibility and Smooth Activations" [arXiv 2022].
Python
17
star
10

ToeffiPy

ToeffiPy is a PyTorch like autograd/deep learning library based only on NumPy.
Python
16
star
11

Dirac-GAN

PyTorch reimplementation of the DiracGAN proposed in the paper "Which Training Methods for GANs do actually Converge?" [ICML 2018].
Python
16
star
12

Optical-Flow-Visualization-PyTorch

PyTorch implementation of the classical optical flow visualization by Baker et al. [ICCV 2007].
Python
13
star
13

HyperMixer

PyTorch reimplementation of the paper "HyperMixer: An MLP-based Green AI Alternative to Transformers" [arXiv 2022].
Python
13
star
14

Multi-StyleGAN

Official and maintained implementation of the paper "Multi-StyleGAN: Towards Image-Based Simulation of Time-Lapse Live-Cell Microscopy" [MICCAI 2021].
Python
10
star
15

DeepFoveaPP_for_Video_Reconstruction_and_Super_Resolution

DeepFovea++: Reconstruction and Super-Resolution for Natural Foveated Rendered Videos (PyTorch).
Python
10
star
16

FNet2D

FNet 2D: Scaling Fourier Transform Token Mixing To Vision
Python
7
star
17

Differentiable_JPEG

This repo reimplements the differentiable JPEG proposed in "JPEG-resistant Adversarial Images".
Python
6
star
18

Yeast-in-Microstructures-Dataset

Official and maintained implementation of the dataset paper "An Instance Segmentation Dataset of Yeast Cells in Microstructures" [EMBC 2023].
Python
6
star
19

Pade-Activation-Unit

PyTorch reimplementation of the paper "Padé Activation Units: End-to-end Learning of Flexible Activation Functions in Deep Networks" [ICLR 2020].
Python
5
star
20

DL4NLP_Cheatsheet_TUD

Cheatsheet for the lecture Deep Learning for Natural Language Processing at TU Darmstadt
3
star
21

3D_Baggage_Segmentation

This repo implements a 3D segmentation task for an airport baggage dataset.
Python
2
star
22

SmeLU-Triton

Triton reimplementation of the Smooth ReLU activation function proposed in the paper "Real World Large Scale Recommendation Systems Reproducibility and Smooth Activations" [arXiv 2022].
Python
2
star
23

simple_logistic_regression

Simple logistic regression model with autograd for Statistics II
Python
1
star
24

CV2_Cheatsheet_TUD

Cheatsheet for the lecture Computer Vision at TU Darmstadt
1
star
25

Latex_Auto_Compile

Short python script for auto compiling Latex files.
Python
1
star
26

Elektronik_Formelsammlung_TUD

Formelsammlung für das Modul Elektronik (TU Darmstadt).
1
star
27

Neural_Network_cpp

Neural network from scratch in C++.
C++
1
star