rust-bert

Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)

rust-tokenizers

Rust-tokenizer offers high-performance tokenizers for modern language models, including WordPiece, Byte-Pair Encoding (BPE) and Unigram (SentencePiece) models

Financial-Causality-Extraction

Fincausal 2020 shared task

SentencePiece-Rust-example

Supporting example for "A Rust SentencePiece implementation"

bpe-example

Supporting code for post on Rust implementation of Byte pair Encoding