conceptofmind/Sparrow-rlhf-pytorch conceptofmind/Sparrow-rlhf-pytorch

  • Stars
    star
    4
  • Rank 3,304,323 (Top 66 %)
  • Language
    Python
  • License
    MIT License
  • Created almost 2 years ago
  • Updated over 1 year ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
conceptofmind/Sparrow-rlhf-pytorch
github

conceptofmind

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

An open-source implementation of DeepMind's Sparrow with RLHF.

More Repositories

1

PaLM

An open-source implementation of Google's PaLM models
Python
805
star
2

LaMDA-rlhf-pytorch

Open-source pre-training implementation of Google's LaMDA in PyTorch. Adding RLHF similar to ChatGPT.
Python
460
star
3

toolformer

Python
336
star
4

t5-pytorch

Implementation of Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer in PyTorch.
Python
46
star
5

flash-gpt

Add Flash-Attention to Huggingface Models
Python
33
star
6

vit-flax

Implementation of numerous Vision Transformers in Google's JAX and Flax.
Python
19
star
7

PaLM-flax

Implementation of the SOTA Transformer architecture from PaLM - Scaling Language Modeling with Pathways in JAX/Flax
Python
14
star
8

PaLM-rlhf-jax

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM. Built in collaboration with Lucidrains.
Python
9
star
9

ViT-Patch-Merger

Python
7
star
10

SemDeDup

An unofficial implementation of SemDeDup: Data-efficient learning at web-scale through semantic deduplication.
Python
6
star
11

Huggingface-deduplicate

Python
5
star
12

hf_fsdp

A very basic fsdp wrapper for HF.
Python
5
star
13

Simple-ViT-flax

Python
4
star
14

flan-Llama

Code for training Llama on the Flan Collection
Python
4
star
15

dsp-langchain

Python
4
star
16

Adaptive-Token-Sampling-Flax

Python
4
star
17

Llama-rlhf-pytorch

3
star
18

Perpetrator

An API for Redteaming large language models.
3
star
19

CaiT-Flax

Python
3
star
20

oig-Llama

Open Instruct fine-tuned Llama
3
star
21

jrlx

Python
2
star
22

Twins-SVT-Flax

An open-source implementation of the Twins: Revisiting the Design of Spatial Attention in Vision Transformers research paper in Google's JAX and Flax.
Python
2
star
23

t8t-pytorch

The official repository for Towards 8-bit Transformers. Stable training of LLMs at mixed 8-bit precision.
2
star
24

WebGPT

Combining LangChain with LLMs to build WebGPT.
2
star
25

BloomCoder

Python
2
star
26

podcasts-are-all-you-need

Python
2
star
27

ViT

Python
1
star
28

Crossformer-flax

Python
1
star
29

DeepViT-flax

Implementation of Deep Vision Transformer in Flax
Python
1
star
30

ViT-haiku

Python
1
star
31

LeViT-flax

Python
1
star
32

OPTCode

Python
1
star
33

bpt-pytorch

Python
1
star
34

deep-reinforcement-learning-go

Creation and implementation of an Alpha Go engine from scratch
Jupyter Notebook
1
star
35

Everything-Machine-Learning

1
star
36

Token-to-Token-ViT-flax

Python
1
star
37

ViT-Small-Datasets-flax

Python
1
star
38

job-posting-analysis

Data Science job posting analysis and resume comparison using NLP and Scikit Learn
Jupyter Notebook
1
star
39

charcuterie

A library for sampling Huggingface datasets.
1
star
40

tokenize_hf

Python
1
star
41

annotated-transformer

Python
1
star
42

Orchestrator

An API for different data cleaning tools.
Python
1
star
43

conceptofmind

Config files for my GitHub profile.
1
star
44

unsloth-bert

Python
1
star