asprenger/ray_vllm_inference asprenger/ray_vllm_inference

  • Stars
    star
    39
  • Rank 691,522 (Top 14 %)
  • Language
    Python
  • License
    Apache License 2.0
  • Created 12 months ago
  • Updated 6 months ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
asprenger/ray_vllm_inference
github

asprenger

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.

More Repositories

1

keras_fc_densenet

Keras implementation of the Fully Convolutional DenseNets for Semantic Segmentation paper.
Jupyter Notebook
9
star
2

doc2vec

Python scripts to train a doc2vec models using Gensim.
Jupyter Notebook
6
star
3

keras_acgan

Train an Auxiliary Classifier Generative Adversarial Network (ACGAN)
Jupyter Notebook
6
star
4

distributed-training-patterns

Experiments with low level communication patterns that are useful for distributed training.
Python
5
star
5

rl-experiments

TensorFlow implementations of different reinforcement learning algorithms.
Python
4
star
6

tf-param-search

Prototyping TensorFlow parameter search
Python
2
star
7

tf-playground

Playground for TensorFlow APIs and components
Python
1
star
8

ml_workshop

"ML Workshop for Developers" content
Jupyter Notebook
1
star
9

tensorboard-embedding-visualization

Embedding Visualization in TensorBoard
Python
1
star
10

ray_retriever

A scalable implementation of a retrieval augmented Q/A assistant based on Ray Serve and Weaviate.
Python
1
star