bentoml/LLMGateway bentoml/LLMGateway

  • Stars
    star
    1
  • Language
    Python
  • Created 4 months ago
  • Updated 4 months ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
bentoml/LLMGateway
github

bentoml

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

More Repositories

1

OpenLLM

Run any open-source LLMs, such as Llama 3.1, Gemma, as OpenAI compatible API endpoint in the cloud.
Python
9,813
star
2

BentoML

The easiest way to serve AI apps and models - Build reliable Inference APIs, LLM apps, Multi-model chains, RAG service, and much more!
Python
7,025
star
3

Yatai

Model Deployment at Scale on Kubernetes πŸ¦„οΈ
TypeScript
788
star
4

BentoDiffusion

BentoDiffusion: A collection of diffusion models served with BentoML
Python
331
star
5

stable-diffusion-server

Deploy Your Own Stable Diffusion Service
Python
196
star
6

bentoctl

Fast model deployment on any cloud πŸš€
Python
175
star
7

gallery

BentoML Example Projects 🎨
Python
134
star
8

BentoVLLM

Self-host LLMs with vLLM and BentoML
Python
64
star
9

OCR-as-a-Service

Turn any OCR models into online inference API endpoint πŸš€ πŸŒ–
Python
49
star
10

CLIP-API-service

CLIP as a service - Embed image and sentences, object recognition, visual reasoning, image classification and reverse image search
Jupyter Notebook
48
star
11

transformers-nlp-service

Online Inference API for NLP Transformer models - summarization, text classification, sentiment analysis and more
Python
43
star
12

llm-bench

Python
28
star
13

rag-tutorials

a series of tutorials implementing rag service with BentoML and LlamaIndex
Python
23
star
14

simple_di

Simple dependency injection framework for Python
Python
21
star
15

BentoChatTTS

Python
21
star
16

Fraud-Detection-Model-Serving

Online model serving with Fraud Detection model trained with XGBoost on IEEE-CIS dataset
Jupyter Notebook
16
star
17

yatai-deployment

πŸš€ Launching Bento in a Kubernetes cluster
Go
16
star
18

google-cloud-run-deploy

Fast model deployment on Google Cloud Run
Python
15
star
19

aws-sagemaker-deploy

Fast model deployment on AWS Sagemaker
Python
15
star
20

aws-lambda-deploy

Fast model deployment on AWS Lambda
Python
14
star
21

aws-ec2-deploy

Fast model deployment on AWS EC2
Python
14
star
22

BentoLMDeploy

Self-host LLMs with LMDeploy and BentoML
Python
14
star
23

yatai-image-builder

🐳 Build OCI images for Bentos in k8s
Go
14
star
24

sentence-embedding-bento

Sentence Embedding as a Service
Jupyter Notebook
14
star
25

IF-multi-GPUs-demo

Python
12
star
26

openllm-models

Python
10
star
27

BentoSVD

Python
10
star
28

BentoWhisperX

Python
10
star
29

diffusers-examples

API serving for your diffusers models
Python
10
star
30

BentoCLIP

building a CLIP application using BentoML
Python
8
star
31

Pneumonia-Detection-Demo

Pneumonia Detection - Healthcare Imaging Application built with BentoML and fine-tuned Vision Transformer (ViT) model
Python
8
star
32

yatai-chart

Helm Chart for installing Yatai on Kubernetes ⎈
Mustache
7
star
33

benchmark

BentoML Performance Benchmark πŸ†š
Jupyter Notebook
7
star
34

BentoTRTLLM

Python
6
star
35

plugins

the swish knife to all things bentoml.
Starlark
6
star
36

bentoctl-operator-template

Python
6
star
37

heroku-deploy

Deploy BentoML bundled models to Heroku
Python
6
star
38

quickstart

BentoML Quickstart Example
Python
6
star
39

BentoSentenceTransformers

how to build a sentence embedding application using BentoML
Python
5
star
40

BentoYolo

BentoML service of YOLO v8
Python
5
star
41

google-compute-engine-deploy

HCL
5
star
42

bentoml-core

Rust
5
star
43

BentoControlNet

Python
4
star
44

BentoBark

Python
4
star
45

BentoRAG

Tutorial: Build RAG Apps with Custom Models Served with BentoML
Python
4
star
46

BentoXTTS

how to build an text-to-speech application using BentoML
Python
4
star
47

containerize-push-action

docker's build-and-push-action equivalent for bentoml
TypeScript
4
star
48

BentoBLIP

how to build an image captioning application on top of a BLIP model with BentoML
Python
3
star
49

deploy-bento-action

A GitHub Action to deploy bento to cloud
3
star
50

azure-functions-deploy

Fast model deployment on Azure Functions
Python
3
star
51

azure-container-instances-deploy

Fast model deployment on Azure container instances
Python
3
star
52

BentoFunctionCalling

Python
3
star
53

llm-router

LLM Router Demo
Python
3
star
54

BentoResnet

Python
2
star
55

bentoml-arize-fraud-detection-workshop

Jupyter Notebook
2
star
56

BentoSDXLTurbo

how to build an image generation application using BentoML
Python
2
star
57

BentoSearch

Search with LLM
Python
2
star
58

BentoInfinity

Python
2
star
59

BentoMLCLLM

Python
2
star
60

yatai-schemas

Go
1
star
61

bentoctl-workshops

Python
1
star
62

bentocloud-homepage-news

1
star
63

yatai-common

Go
1
star
64

BentoMoirai

Python
1
star
65

.github

βœ¨πŸ±πŸ¦„οΈ
1
star
66

bentoml-unsloth

BentoML Unsloth integration
Python
1
star
67

BentoShield

Python
1
star
68

BentoTGI

Python
1
star
69

openllm-benchmark

Python
1
star