kuprel/minbpe-pytorch kuprel/minbpe-pytorch

  • Stars
    star
    34
  • Rank 766,985 (Top 16 %)
  • Language
    Python
  • License
    MIT License
  • Created 9 months ago
  • Updated 9 months ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
kuprel/minbpe-pytorch
github

kuprel

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization, with PyTorch/CUDA

More Repositories

1

min-dalle

min(DALL·E) is a fast, minimal port of DALL·E Mini to PyTorch
Python
3,487
star
2

pycaffe-recurrent

IPython notebook for training multilayer LSTM and RNN networks with pycaffe
Python
53
star
3

min-dalle-flax

This contains the Flax model of min(DALL·E) and code for converting it to PyTorch
Python
46
star
4

ppg_plugin

Java
5
star
5

thesis

TeX
1
star