@EleutherAI
Guthub

EleutherAI

Twitter logo Share LinkedIn logo Share Facebook logo Share

Top repositories

1

gpt-neo

An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.
Python
8,150
star
2

gpt-neox

An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.
Python
6,467
star
3

lm-evaluation-harness

A framework for few-shot evaluation of language models.
Python
5,205
star
4

pythia

The hub for EleutherAI's work on interpretability and learning dynamics
Jupyter Notebook
1,935
star
5

the-pile

Python
1,375
star
6

math-lm

Python
975
star
7

polyglot

Polyglot: Large Language Models of Well-balanced Competence in Multi-languages
460
star
8

DALLE-mtf

Open-AI's DALL-E for large scale training in mesh-tensorflow.
Python
436
star
9

vqgan-clip

Jupyter Notebook
339
star
10

concept-erasure

Erasing concepts from neural representations with provable guarantees
Python
186
star
11

elk

Keeping language models honest by directly eliciting knowledge encoded in their activations.
Python
171
star
12

oslo

OSLO: Open Source for Large-scale Optimization
Python
170
star
13

lm_perplexity

Python
137
star
14

knowledge-neurons

A library for finding knowledge neurons in pretrained transformer models.
Python
130
star
15

cookbook

Deep learning for dummies. All the practical details and useful utilities that go into working with real models.
Python
123
star
16

pyfra

Python Research Framework
Python
107
star
17

dps

Data processing system for polyglot
Python
83
star
18

openwebtext2

Python
81
star
19

info

(Deprecated) A hub for onboarding & other information.
78
star
20

project-menu

See the issue board for the current status of active and prospective projects!
65
star
21

stackexchange-dataset

Python tools for processing the stackexchange data dumps into a text dataset for Language Models
Python
64
star
22

magiCARP

One stop shop for all things carp
Python
58
star
23

tqdm-multiprocess

Using queues, tqdm-multiprocess supports multiple worker processes, each with multiple tqdm progress bars, displaying them cleanly through the main process. It offers similar functionality for python logging.
Python
41
star
24

aria

Python
36
star
25

semantic-memorization

Jupyter Notebook
34
star
26

hae-rae

30
star
27

improved-t5

Experiments for efforts to train a new and improved t5
Python
26
star
28

features-across-time

Understanding how features learned by neural networks evolve throughout training
Python
25
star
29

mp_nerf

Massively-Parallel Natural Extension of Reference Frame
Jupyter Notebook
25
star
30

pile-pubmedcentral

A script for collecting the PubMed Central dataset in a language modelling friendly format.
Python
20
star
31

best-download

URL downloader supporting checkpointing and continuous checksumming.
Python
19
star
32

polyglot-data

data related codebase for polyglot project
Python
19
star
33

elk-generalization

Investigating the generalization behavior of LM probes trained to predict truth labels: (1) from one annotator to another, and (2) from easy questions to hard
Jupyter Notebook
19
star
34

text-generation-testing-ui

Web app for demoing the EAI models
JavaScript
16
star
35

exploring-contrastive-topology

Jupyter Notebook
16
star
36

rnngineering

Engineering the state of RNN language models (Mamba, RWKV, etc.)
Jupyter Notebook
16
star
37

mdl

Minimum Description Length probing for neural network representations
Python
14
star
38

pile_dedupe

Pile Deduplication Code
Python
14
star
39

pilev2

Python
13
star
40

distilling

Experiments with distilling large language models.
Python
13
star
41

lm-eval2

Python
11
star
42

equivariance

A framework for implementing equivariant DL
Jupyter Notebook
10
star
43

radioactive-lab

Adapting the "Radioactive Data" paper to work for text models
Python
9
star
44

tagged-pile

Part-of-Speech Tagging for the Pile and RedPajama
Python
9
star
45

pile-literotica

Download, parse, and filter data from Literotica. Data-ready for The-Pile.
Python
8
star
46

hn-scraper

Python
8
star
47

multimodal-fid

Python
7
star
48

pile-cc-filtering

The code used to filter CC data for The Pile
Python
6
star
49

minetest-baselines

Baseline agents for Minetest tasks.
Python
6
star
50

CodeCARP

Data collection pipeline for CodeCARP. Includes PyCharm plugins.
6
star
51

LLM-Markov-Chains

Project github for LLM Markov Chains Project
6
star
52

pile-uspto

A script for collecting the USPTO Backgrounds dataset in a language modelling friendly format.
Python
6
star
53

thonkenizers

yes
5
star
54

minetest-interpretabilty-notebook

Jupyter notebook for the interpretablity section of the minetester blog post
Jupyter Notebook
5
star
55

visual-grounding

Visually ground GPT-Neo 1.3b and 2.7b
Python
5
star
56

Unpaired-Image-Generation

Project Repo for Unpaired Image Generation project
5
star
57

pile-enron-emails

A script for collecting the Enron Emails dataset in a language modelling friendly format.
Python
5
star
58

architecture-experiments

Repository to host architecture experiments and development using Paxml and Praxis
Python
5
star
59

llemma-sample-explorer

Sample explorer tool for the Llemma models.
HTML
5
star
60

pile-explorer

For exploring the data and documenting its limitations
Python
4
star
61

lm-scope

Jupyter Notebook
4
star
62

megatron-3d

Python
4
star
63

tokengrams

Efficiently computing & storing token n-grams from large corpora
Rust
4
star
64

ccs

Python
4
star
65

latent-video-diffusion

Latent video diffusion
Python
3
star
66

eleutherai-instruct-dataset

A large instruct dataset for open-source models (WIP).
3
star
67

isaac-mchorse

EleutherAI's discord bot
Python
3
star
68

pile-allpoetry

Scraper to gather poems from allpoetry.com
Python
3
star
69

eai-prompt-gallery

Library of interesting prompt generations
JavaScript
3
star
70

eleutherai.github.io

This is the Hugo generated website for eleuther.ai. The source of this build is new-website repo.
HTML
3
star
71

website

New website for EleutherAI based on Hugo static site generator
HTML
3
star
72

variance-across-time

Studying the variance in neural net predictions across training time
Python
3
star
73

pile-ubuntu-irc

A script for collecting the Ubuntu IRC dataset in a language modelling friendly format.
Python
3
star
74

aria-amt

MIDI conditioned automatic music transcription
Jupyter Notebook
3
star
75

reddit-comment-processing

Python
2
star
76

language-adaptation

2
star
77

EvilModel

A replication of "EvilModel 2.0: Bringing Neural Network Models into Malware Attacks"
2
star
78

bucket-cleaner

A small utility to clear out old model checkpoints in Google Cloud Buckets whilst keeping tensorboard event files
Python
2
star
79

groupoid-rl

Jupyter Notebook
2
star
80

lang-filter

Filter text files or archives by language
Python
1
star
81

eleuther-blog

here is the generated content for the EleutherAI blog. Source is from new-website repo
HTML
1
star
82

prefix-free-tokenizer

A prefix free tokenizer
Python
1
star
83

alignment-reader

Search and filter through alignment literature
JavaScript
1
star
84

grouch

HTML
1
star
85

perceptors

central location for access to pretrained models for CLIP and variants, with common API and out-of-the-box differentiable weighted multi-perceptor
1
star
86

classifier-latent-diffusion

Python
1
star
87

common-llm-settings

Common LLM Settings App
JavaScript
1
star
88

bayesian-adam

Exactly what it says on the tin
Python
1
star
89

pile-cord19

A script for collecting the CORD-19 dataset in a language modelling friendly format.
Python
1
star
90

conceptual-constraints

Applying LEACE to models during training
Jupyter Notebook
1
star
91

truncated-gaussian

Method-of-moments estimation and sampling for truncated multivariate Gaussian distributions
Python
1
star