gautierdag/bpeasy gautierdag/bpeasy

  • Stars
    star
    129
  • Rank 277,608 (Top 6 %)
  • Language
    Python
  • License
    MIT License
  • Created 11 months ago
  • Updated 9 months ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
gautierdag/bpeasy
github

gautierdag

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Fast bare-bones BPE for modern tokenizer training

More Repositories

1

pytorch-attentive-lm

Pytorch Implementation of an A-RNN-LM
Python
13
star
2

tokenizer-bench

Code for the paper "Getting the most out of your tokenizer for pre-training and domain adaptation"
Python
9
star
3

cultural-evolution-engine

This is a first take at a task and model agnostic CEE
Jupyter Notebook
4
star
4

pcfg-attention

Using an Attention Seq-to-Seq model on pcfg
Python
3
star
5

pytorch-research-template

Opinionated Research Template
Python
3
star
6

batche

Batch cache decorator
Python
2
star
7

riiid

Repo for RIIID Kaggle competition
Jupyter Notebook
2
star
8

rsa-a

A deeper dive into Representational Similarity Analysis in Referential Games
Jupyter Notebook
2
star
9

learnability-communication-tradeoff

learnability-communication-tradeoff
Jupyter Notebook
2
star
10

pythondatastruc

Python Data Structures - simple implementations
Python
1
star
11

serverless-plugin-ts-validation

Serverless plugin to generate validation JSON schemas from Typescript interfaces
TypeScript
1
star
12

plancraft

Plancraft is a minecraft environment and agent that innovates on planning LLM agents with a retriever
Jupyter Notebook
1
star
13

awesome-constrained-decoding

List of open-source constrained decoding projects
1
star