Discover lucidrains/alphafold3-pytorch Open Source project

Stars
483
Rank 91,070 (Top 2 %)
Language
Python
License
MIT License
Created 6 months ago
Updated 5 months ago

lucidrains

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Implementation of Alphafold 3 in Pytorch

vit-pytorch

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

Python

13,633

DALLE2-pytorch

Implementation of DALL-E 2, OpenAI's updated text-to-image synthesis neural network, in Pytorch

Python

11,068

imagen-pytorch

Implementation of Imagen, Google's Text-to-Image Neural Network, in Pytorch

Python

7,832

PaLM-rlhf-pytorch

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

Python

7,611

DALLE-pytorch

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

Python

5,132

deep-daze

Simple command line tool for text to image generation using OpenAI's CLIP and Siren (Implicit neural representation network). Technique was originally created by https://twitter.com/advadnoun

Python

4,387

denoising-diffusion-pytorch

Implementation of Denoising Diffusion Probabilistic Model in Pytorch

Python

3,959

stylegan2-pytorch

Simplest working implementation of Stylegan2, state of the art generative adversarial network, in Pytorch. Enabling everyone to experience disentanglement

Python

3,433

musiclm-pytorch

Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch

Python

3,048

x-transformers

A simple but complete full-attention transformer with a set of promising experimental features from various papers

Python

2,707

big-sleep

A simple command line tool for text to image generation, using OpenAI's CLIP and a BigGAN. Technique was originally created by https://twitter.com/advadnoun

Python

2,446

audiolm-pytorch

Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch

Python

2,285

lion-pytorch

🦁 Lion, new optimizer discovered by Google Brain using genetic algorithms that is purportedly better than Adam(w), in Pytorch

Python

1,933

toolformer-pytorch

Implementation of Toolformer, Language Models That Can Use Tools, by MetaAI

Python

1,905

reformer-pytorch

Reformer, the efficient Transformer, in Pytorch

Python

1,870

make-a-video-pytorch

Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch

Python

1,853

gigagan-pytorch

Implementation of GigaGAN, new SOTA GAN out of Adobe. Culmination of nearly a decade of research into GANs

Python

1,632

alphafold2

To eventually become an unofficial Pytorch implementation / replication of Alphafold2, as details of the architecture get released

Python

1,536

lightweight-gan

Implementation of 'lightweight' GAN, proposed in ICLR 2021, in Pytorch. High resolution image generations that can be trained within a day or two

Python

1,526

lambda-networks

Implementation of LambdaNetworks, a new approach to image recognition that reaches SOTA with less compute

Python

1,516

byol-pytorch

Usable Implementation of "Bootstrap Your Own Latent" self-supervised learning, from Deepmind, in Pytorch

Python

1,497

self-rewarding-lm-pytorch

Implementation of the training framework proposed in Self-Rewarding Language Model, from MetaAI

Python

1,253

naturalspeech2-pytorch

Implementation of Natural Speech 2, Zero-shot Speech and Singing Synthesizer, in Pytorch

Python

1,214

flamingo-pytorch

Implementation of 🦩 Flamingo, state-of-the-art few-shot visual question answering attention net out of Deepmind, in Pytorch

Python

1,155

video-diffusion-pytorch

Implementation of Video Diffusion Models, Jonathan Ho's new paper extending DDPMs to Video Generation - in Pytorch

Python

1,141

soundstorm-pytorch

Implementation of SoundStorm, Efficient Parallel Audio Generation from Google Deepmind, in Pytorch

Python

1,130

CoCa-pytorch

Implementation of CoCa, Contrastive Captioners are Image-Text Foundation Models, in Pytorch

Python

990

performer-pytorch

An implementation of Performer, a linear attention-based transformer, in Pytorch

Python

937

perceiver-pytorch

Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch

Python

935

RETRO-pytorch

Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch

Python

835

mlp-mixer-pytorch

An All-MLP solution for Vision, from Google AI

Python

833

muse-maskgit-pytorch

Implementation of Muse: Text-to-Image Generation via Masked Generative Transformers, in Pytorch

Python

821

PaLM-pytorch

Implementation of the specific Transformer architecture from PaLM - Scaling Language Modeling with Pathways

Python

812

vector-quantize-pytorch

Vector Quantization, in Pytorch

Python

810

phenaki-pytorch

Implementation of Phenaki Video, which uses Mask GIT to produce text guided videos of up to 2 minutes in length, in Pytorch

Python

724

x-clip

A concise but complete implementation of CLIP with various experimental improvements from recent papers

Python

658

bottleneck-transformer-pytorch

Implementation of Bottleneck Transformer in Pytorch

Python

632

memorizing-transformers-pytorch

Implementation of Memorizing Transformers (ICLR 2022), attention net augmented with indexing and retrieval of memories using approximate nearest neighbors, in Pytorch

Python

614

TimeSformer-pytorch

Implementation of TimeSformer from Facebook AI, a pure attention-based solution for video classification

Python

613

MEGABYTE-pytorch

Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch

Python

594

meshgpt-pytorch

Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch

Python

564

nuwa-pytorch

Implementation of NÜWA, state of the art attention network for text to video synthesis, in Pytorch

Python

531

voicebox-pytorch

Implementation of Voicebox, new SOTA Text-to-speech network from MetaAI, in Pytorch

Python

521

point-transformer-pytorch

Implementation of the Point Transformer layer, in Pytorch

Python

518

parti-pytorch

Implementation of Parti, Google's pure attention-based text-to-image neural network, in Pytorch

Python

509

tab-transformer-pytorch

Implementation of TabTransformer, attention network for tabular data, in Pytorch

Python

485

linear-attention-transformer

Transformer based on a variant of attention that is linear complexity in respect to sequence length

Python

468

magvit2-pytorch

Implementation of MagViT2 Tokenizer in Pytorch

Python

436

ema-pytorch

A simple way to keep track of an Exponential Moving Average (EMA) version of your pytorch model

Python

408

egnn-pytorch

Implementation of E(n)-Equivariant Graph Neural Networks, in Pytorch

Python

400

g-mlp-pytorch

Implementation of gMLP, an all-MLP replacement for Transformers, in Pytorch

Python

391

recurrent-memory-transformer-pytorch

Implementation of Recurrent Memory Transformer, Neurips 2022 paper, in Pytorch

Python

384

ring-attention-pytorch

Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch

Python

380

siren-pytorch

Pytorch implementation of SIREN - Implicit Neural Representations with Periodic Activation Function

Python

377

enformer-pytorch

Implementation of Enformer, Deepmind's attention network for predicting gene expression, in Pytorch

Python

352

iTransformer

Unofficial implementation of iTransformer - SOTA Time Series Forecasting using Attention networks, out of Tsinghua / Ant group

Python

349

robotic-transformer-pytorch

Implementation of RT1 (Robotic Transformer) in Pytorch

Python

346

memory-efficient-attention-pytorch

Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attention Does Not Need O(n²) Memory"

Python

342

FLASH-pytorch

Implementation of the Transformer variant proposed in "Transformer Quality in Linear Time"

Python

334

bit-diffusion

Implementation of Bit Diffusion, Hinton's group's attempt at discrete denoising diffusion, in Pytorch

Python

313

medical-chatgpt

Implementation of ChatGPT, but tailored towards primary care medicine, with the reward being able to collect patient histories in a thorough and efficient manner and come up with a reasonable differential diagnosis

Python

311

slot-attention

Implementation of Slot Attention from GoogleAI

Python

303

q-transformer

Implementation of Q-Transformer, Scalable Offline Reinforcement Learning via Autoregressive Q-Functions, out of Google Deepmind

Python

293

BS-RoFormer

Implementation of Band Split Roformer, SOTA Attention network for music source separation out of ByteDance AI Labs

Python

289

classifier-free-guidance-pytorch

Implementation of Classifier Free Guidance in Pytorch, with emphasis on text conditioning, and flexibility to include multiple text embedding models

Python

282

transformer-in-transformer

Implementation of Transformer in Transformer, pixel level attention paired with patch level attention for image classification, in Pytorch

Python

277

axial-attention

Implementation of Axial attention - attending to multi-dimensional data efficiently

Python

273

conformer

Implementation of the convolutional module from the Conformer paper, for use in Transformers

Python

272

mixture-of-experts

A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models

Python

264

deformable-attention

Implementation of Deformable Attention in Pytorch from the paper "Vision Transformer with Deformable Attention"

Python

258

magic3d-pytorch

Implementation of Magic3D, Text to 3D content synthesis, in Pytorch

Python

258

x-unet

Implementation of a U-net complete with efficient attention as well as the latest research findings

Python

252

routing-transformer

Fully featured implementation of Routing Transformer

Python

251

Adan-pytorch

Implementation of the Adan (ADAptive Nesterov momentum algorithm) Optimizer in Pytorch

Python

245

spear-tts-pytorch

Implementation of Spear-TTS - multi-speaker text-to-speech attention network, in Pytorch

Python

241

st-moe-pytorch

Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch

Python

237

perfusion-pytorch

Implementation of Key-Locked Rank One Editing, from Nvidia AI

Python

229

equiformer-pytorch

Implementation of the Equiformer, SE3/E3 equivariant attention network that reaches new SOTA, and adopted for use by EquiFold for protein folding

Python

227

segformer-pytorch

Implementation of Segformer, Attention + MLP neural network for segmentation, in Pytorch

Python

227

sinkhorn-transformer

Sinkhorn Transformer - Practical implementation of Sparse Sinkhorn Attention

Python

222

pixel-level-contrastive-learning

Implementation of Pixel-level Contrastive Learning, proposed in the paper "Propagate Yourself", in Pytorch

Python

220

lumiere-pytorch

Implementation of Lumiere, SOTA text-to-video generation from Google Deepmind, in Pytorch

Python

216

local-attention

An implementation of local windowed attention for language modeling

Python

216

CoLT5-attention

Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch

Python

216

natural-speech-pytorch

Implementation of the neural network proposed in Natural Speech, a text-to-speech generator that is indistinguishable from human recordings for the first time, from Microsoft Research

Python

215

soft-moe-pytorch

Implementation of Soft MoE, proposed by Brain's Vision team, in Pytorch

Python

211

se3-transformer-pytorch

Implementation of SE3-Transformers for Equivariant Self-Attention, in Pytorch. This specific repository is geared towards integration with eventual Alphafold2 replication.

Python

211

block-recurrent-transformer-pytorch

Implementation of Block Recurrent Transformer - Pytorch

Python

205

Mega-pytorch

Implementation of Mega, the Single-head Attention with Multi-headed EMA architecture that currently holds SOTA on Long Range Arena

Python

201

simple-hierarchical-transformer

Experiments around a simple idea for inducing multiple hierarchical predictive model within a GPT

Python

198

med-seg-diff-pytorch

Implementation of MedSegDiff in Pytorch - SOTA medical segmentation using DDPM and filtering of features in fourier space

Python

195

triton-transformer

Implementation of a Transformer, but completely in Triton

Python

195

jax2torch

Use Jax functions in Pytorch

Python

194

flash-cosine-sim-attention

Implementation of fused cosine similarity attention in the same style as Flash Attention

Cuda

194

halonet-pytorch

Implementation of the 😇 Attention layer from the paper, Scaling Local Self-Attention For Parameter Efficient Visual Backbones

Python

193

attention

This repository will house a visualization that will attempt to convey instant enlightenment of how Attention works to someone not working in artificial intelligence, with 3Blue1Brown as inspiration

HTML

189

recurrent-interface-network-pytorch

Implementation of Recurrent Interface Network (RIN), for highly efficient generation of images and video without cascading networks, in Pytorch

Python

188

electra-pytorch

A simple and working implementation of Electra, the fastest way to pretrain language models from scratch, in Pytorch

Python

186

PaLM-jax

Implementation of the specific Transformer architecture from PaLM - Scaling Language Modeling with Pathways - in Jax (Equinox framework)

Python

184

100

unet-stylegan2

A Pytorch implementation of Stylegan2 with UNet Discriminator

Python

182

lucidrains/alphafold3-pytorch

lucidrains

Reviews

Repository Details

More Repositories