• Stars
    star
    220
  • Rank 180,422 (Top 4 %)
  • Language
  • Created over 1 year ago
  • Updated 9 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

🦀 A curated list of Rust tools, libraries, and frameworks for working with LLMs, GPT, AI

                                                                      __       __ __          
.---.-.--.--.--.-----.-----.-----.--------.-----.___.----.--.--.-----|  |_ ___|  |  .--------.
|  _  |  |  |  |  -__|__ --|  _  |        |  -__|___|   _|  |  |__ --|   _|___|  |  |        |
|___._|________|_____|_____|_____|__|__|__|_____|   |__| |_____|_____|____|   |__|__|__|__|__|

https://github.com/jondot/awesome-rust-llm

Awesome Rust LLM is an awesome style list that keeps track and curates the best Rust based LLM frameworks, libraries, tools, tutorials, articles and more. PRs are welcome!

Models & Inference

  • llm - a Rust library for running inference from a number of supported LLMs, loads ggml based models
  • rust-bert - all in one pipelines for for transformer-based models (BERT, DistilBERT, GPT2,...). Good for local embedding, port of transformers (python)
  • llm-chain - chaining LLMs in Rust
  • smartgpt (how it works)- use LLMs with the ability to complete complex tasks using plugins
  • diffusers - Stable Diffusion using Rust, 45% faster than Pytorch
  • postgresml - an amazing Postgres extension to do model fetching, inference all with SQL as part of your Postgres instance

Projects

  • aichat - a pure Rust CLI implementing AI chat, with advanced features such as real-time streaming, text highlighting and more

  • browser-agent - a headless browser driven by GPT-4. Sends off a simplified page representation and receives & executes instructions from GPT using a custom message format

  • tenere - TUI interface for LLMs

How it works? here's a prompt snippet:

You must respond with ONLY one of the following commands AND NOTHING ELSE:
    - CLICK X - click on a given element. You can only click on links, buttons, and inputs!
    - TYPE X \"TEXT\" - type the specified text into the input with id X and press ENTER
    - ANSWER \"TEXT\" - Respond to the user with the specified text once you have completed the objective
  • ajeto - LLM personal assistant
  • shortgpt - Ask shortgpt for instant and concise answers
  • autorust - macros that generate AI driven rust code in compile time
  • clerk - LLM based file organizer
  • gptcommit - prepare commit message with GPT

LLM Memory

  • indexify - A retrieval and long term memory service for LLMs
  • memex - Super-simple, fully Rust powered "memory" (doc store + semantic search) for LLM projects, semantic search, etc.
  • motorhead - a memory and information retrieval server for LLMs.

Core Libraries

  • tiktoken - tiktoken is a Python library with a Rust core implementing a fast BPE tokeniser for use with OpenAI's models

    • BPE is done in Rust
    • Made by OpenAI
  • tiktoken-rs - a Rust focused library based on the tiktoken core with additional enhancements for use in Rust code. (Pyton parts in tiktoken done in pure Rust)

// Rust
use tiktoken_rs::p50k_base;

let bpe = p50k_base().unwrap();
let tokens = bpe.encode_with_special_tokens(
  "This is a sentence   with spaces"
);
println!("Token count: {}", tokens.len());
  • polars - a faster, pure Rust pandas alternative

  • rllama - a pure Rust implemenation of LLaMa inference. Great for embedding into other apps or wrapping for a scripting language.

  • whatlang - Rust library using a multiclass logistic regression model to detect languages

  • OpenAI API - a strongly typed Rust client for the OpenAI API

Tools

  • spider - crawler / spider written in Rust for when you need a whole-website dump. Unlike Scrapy, focuses on dumping data but the post-processing is done later.

Services

  • dust - a full service for workflow running with composable blocks. Core is in Rust, various frontends in Typescript.

Vector Stores

  • pgvecto.rs - Vector database plugin for Postgres, written in Rust, specifically designed for LLM. 20x faster than pgvector
  • qdrant - Qdrant - Vector Database for the next generation of AI applications

More Repositories

1

awesome-react-native

Awesome React Native components, news, tools, and learning material!
JavaScript
33,342
star
2

hygen

The simple, fast, and scalable code generator that lives in your project.
JavaScript
5,610
star
3

graphene

Graphene is a realtime dashboard & graphing toolkit based on D3 and Backbone.
CSS
2,881
star
4

react-flight

The best way to build animation compositions for React.
JavaScript
2,825
star
5

awesome-devenv

A curated list of awesome tools, resources and workflow tips making an awesome development environment.
2,622
star
6

sneakers

A fast background processing framework for Ruby and RabbitMQ
Ruby
2,246
star
7

goweight

A tool to analyze and troubleshoot a Go binary size.
Go
1,687
star
8

rust-how-do-i-start

Hand curated advice and pointers for getting started with Rust
1,055
star
9

ReactNativeKatas

This is a project that lets you participate in a fully-immersive, hands-on, and fun learning experience for React Native.
JavaScript
947
star
10

awesome-weekly

An "awesome" type curated list of quality weekly subscription newsletters from the software world
890
star
11

blade

Better asset workflow for iOS developers. Generate Xcode image catalogs for iOS / OSX app icons, universal images, and more.
Go
818
star
12

groundcontrol

Manage and monitor your Raspberry Pi with ease
Go
769
star
13

rn-snoopy

Snoopy is a profiling tool for React Native, that lets you snoop on the React Native Bridge.
JavaScript
523
star
14

react-native-slowlog

A high-performance timer based profiler for React Native that helps you track big performance problems.
JavaScript
374
star
15

crunch

A fast to develop, fast to run, Go based toolkit for ETL and feature extraction on Hadoop.
Go
210
star
16

moxy

The programmable mock proxy
Ruby
120
star
17

webnull

web/null eats your HTTP
CoffeeScript
104
star
18

padrino-warden

A Padrino module that provides authentication for your Padrino application through Warden
Ruby
100
star
19

tauri-tray-app

A Tauri tray app starter 🦀
TypeScript
94
star
20

roundtrip

Simple tracking and metrics for your business processes in real-time
Ruby
89
star
21

awesome-aha

Awesome list for "Aha!" moments related to programming and computer science. Accelerate your learning.
82
star
22

nina

The friendly web microframework that performs!. Nina is a feature complete web microframework for the .Net platform, inspired by Sinatra
C#
70
star
23

frenzy_bunnies

RabbitMQ JRuby based worker framework on top of march_hare (hot_bunnies)
Ruby
70
star
24

redux-stack

Redux Stack is a library that helps you build modular, structured, and cleaner redux apps
JavaScript
67
star
25

storybook-cartesian

Automatically generate stories for all of your component variants
TypeScript
63
star
26

vitals

Flexible StatsD instrumentation for Rails, Rack, Grape and more
Ruby
51
star
27

version_bumper

Bump your versions
Ruby
47
star
28

logbook

Log your memories onto virtual logbooks made of Gists
Ruby
43
star
29

mediumize

Automatically post (and cross-post) your markdown style blog posts to your Medium account from Jekyll, Middleman, Hugo and others.
Ruby
40
star
30

nchurn

.Net based churn analyzer for your build
C#
37
star
31

awesome-designops

Awesome DesignOps is an awesome style list that curates the best design ops news, tools, tutorials, articles and more.
JavaScript
37
star
32

scatter

Ruby
35
star
33

benchmark-ipsa

An iterations per second enhancement to Benchmark that includes memory allocations
Ruby
32
star
34

go-cli-starter

A Go based command line interface starter app (CLI)
Go
32
star
35

fattyproject

Go
32
star
36

react-native-network-boot

An alternative way of bootstrapping development network bundling for React Native
JavaScript
31
star
37

awesome-beginners

A list that curates resources to help you teach your kids, wives, husbands, family or friends how to code
29
star
38

gulpjs-phaser

CSS
29
star
39

xtaskops

Goodies for working with the xtask concept in Rust
Rust
29
star
40

PrimerApp

JavaScript
25
star
41

groundcontrol-ui

JavaScript
25
star
42

attrs-serde

A serialization addon for attrs.
Python
24
star
43

passage

Personal, tiny, flexible, OpenID provider
Ruby
22
star
44

pgpipeline

A Scrapy pipeline module to persist items to a postgres table automatically.
Python
21
star
45

hygen-CRA

Perl 6
21
star
46

redux-duet

Redux action and handlers together, alleviate boilerplate.
JavaScript
20
star
47

langchain-llm-katas

This is a an open-source project designed to help you improve your skills with AI engineering using LLMs and the langchain library
Python
18
star
48

vscode-hygen

This extension bundles Hygen into VSCode and offers seamless code generator functionality right into your editor.
TypeScript
18
star
49

pcwr

Pragmatic Concurrency With Ruby
Ruby
17
star
50

castbox

A chromecast 1.0 emulator
Go
17
star
51

formation

A generic functional middleware infrastructure for Python.
Python
17
star
52

elb-dash

A self-updating ELB status board / dashboard built with React, Coffeescript and Node.js.
CoffeeScript
16
star
53

make-vscode-more-like-vim

Make VSCode more like vim
15
star
54

qtools

qtools are a set of tools to greatly ease your MSMQ operations, monitoring and deployment
C#
15
star
55

hygen-add

hygen-add is one of the tools in the Hygen toolbelt which allows you to add pre-made generator packages to your project.
JavaScript
14
star
56

deep-learning-parameters-cheatsheet

14
star
57

elasticsearch-balance

Visualize the data distribution of your Elastic Search cluster using a Treemap
JavaScript
13
star
58

darkness

JRuby with Swing WriteRoom/Darkroom-like editor.
Java
13
star
59

heatmapdotnet

Heatmap generator for .Net
C#
13
star
60

blade-sample

Swift
13
star
61

rawsort

A simple but powerful RAW photo import tool built to have amazing performance and integrate with photography workflows.
Rust
13
star
62

hypermatch

A fast, sandboxed micro matching engine with serializable rules.
JavaScript
12
star
63

cottonballs

Your own mock, fluffy version of GCM for testing purposes
CoffeeScript
12
star
64

packs

An open-source BoxJS/CSS/Etc clone, suitable for hosting at Heroku
JavaScript
12
star
65

rrgen

Rust
11
star
66

10bisbar

A bitbar plugin that helps you do the math for your 10bis account
Go
11
star
67

primer-bind

Go
11
star
68

dg

Check a folder for dirty git repositories, forgotten branches and commits
Rust
11
star
69

lidar

A take on ThoughtWorks' Radar. You can use this tool and system to make your very own technological Radar.
JavaScript
11
star
70

statsd-stack

A statsd Sprinkle stack
Ruby
11
star
71

react-rust-chrome-starter

Chrome extension template with Rust 🦀 and React using Vite and tailwind
TypeScript
11
star
72

celeste

An all-in-one tool for the repository maintainer.
JavaScript
10
star
73

statsd-cli

Simple Statsd command line interface
Ruby
10
star
74

jill

Jill is your README.md assistant
Ruby
10
star
75

dazzling

Dazzling is a project website generator based on Gatsby and React that's simple, quick, and extensible.
JavaScript
10
star
76

logolang.org

Retro LOGO with a modern twist. Built with Rust+WASM 🦀
TypeScript
10
star
77

firebase-simple-storage

An unofficial Go based client for Firebase storage that preserves security (goes through Firebase and not Google Cloud Storage).
Go
10
star
78

signup

Easy and lightweight signup service
Go
9
star
79

keymaps

Shell
9
star
80

gravy

Sweet sauce for your Gravatars: realtime gravatar manipulation with Node.JS
9
star
81

autobrew

Automate homebrew formula publishing for your CLI tools, regardless of programming language.
Go
9
star
82

webogram

Snap your web pages through Instagram inspired filters. You can use this to generate wallpapers for phone and desktop, icons, and book covers
JavaScript
9
star
83

goddamn-javascript-babel

A zero config package that gives you a bleeding edge Javascript, with the latest proposals including pipe operator foo |> bar, optional chaining foo?.bar, null coalescing foo ?? bar and more.
JavaScript
9
star
84

arper

A network utility and library for discovering network device via ARP scans, including vendor names.
Go
8
star
85

react-native-fs-walker

A React Native file system walker and file tree debugging tool
JavaScript
8
star
86

hypercontroller

A more civilized controller abstraction for TypeScript and Node.js. Hosts libraries for controllers built using hypergen
TypeScript
8
star
87

es-diag

Elastic Search health checks and problem discovery toolkit
Ruby
8
star
88

pocket-emails

A compact tool that sends digest emails from your Pocket account with Mailgun.
JavaScript
8
star
89

dash

Communicate to the mass with Dash on your TV/LCD/Plasma
JavaScript
8
star
90

armor

WIP - a pragmatic Go microservice framwork
Go
8
star
91

react-native-bundles

Hand-picked bundles of React Native libraries and components that go well together for any kind of app
8
star
92

react-redux-classconnect

JavaScript
7
star
93

react-native-group-image

A <GroupImage /> component for React Native. A group image will be responsive towards the amount of images it needs to display
JavaScript
7
star
94

serverless-zen

A serverless starter project focusing on uncompromising local development experience without serverless fatigue.
TypeScript
7
star
95

dotlinker

Use dotrunner and dotlinker to build a fantastically aesthetic macOS dotfiles set up.
Python
7
star
96

dotfiles

configuration files.
Vim Script
7
star
97

outpostapp

Flexible file upload app
JavaScript
7
star
98

appium-solo

Appium simplified: Run E2E tests with a dedicated appium instance per device.
JavaScript
6
star
99

stylomatic

Zero configuration for typescript, react, and more
JavaScript
6
star
100

mongomon

A Python mongodb monitor and profiler for development.
Python
6
star