server
The Triton Inference Server provides an optimized cloud and edge inferencing solution.pytriton
PyTriton is a Flask/FastAPI-like interface that simplifies Triton's deployment in Python environments.client
Triton Python, C++ and Java client libraries, and GRPC-generated client examples for go, java and scala.python_backend
Triton backend that enables pre-process, post-processing and other logic to be implemented in Python.tensorrtllm_backend
The Triton TensorRT-LLM Backendfastertransformer_backend
tutorials
This repository contains tutorials and examples for Triton Inference Servermodel_analyzer
Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Server models.backend
Common source, scripts and utilities for creating Triton backends.model_navigator
Triton Model Navigator is a tool that provides the ability to automate the process of model deployment on the Triton Inference Server.dali_backend
The Triton backend that allows running GPU-accelerated data pre-processing pipelines implemented in DALI's python API.onnxruntime_backend
The Triton backend for the ONNX Runtime.pytorch_backend
The Triton backend for the PyTorch TorchScript models.vllm_backend
core
The core library and APIs implementing the Triton Inference Server.fil_backend
FIL backend for the Triton Inference Servercommon
Common source, scripts and utilities shared across all Triton repositories.hugectr_backend
tensorrt_backend
The Triton backend for TensorRT.tensorflow_backend
The Triton backend for TensorFlow.paddlepaddle_backend
openvino_backend
OpenVINO backend for Triton.developer_tools
stateful_backend
Triton backend for managing the model state tensors automatically in sequence batchercontrib
Community contributions to Triton that are not officially supported or maintained by the Triton project.third_party
Third-party source packages that are modified for use in Triton.checksum_repository_agent
The Triton repository agent that verifies model checksums.identity_backend
Example Triton backend that demonstrates most of the Triton Backend API.redis_cache
TRITONCACHE implementation of a Redis cacherepeat_backend
An example Triton backend that demonstrates sending zero, one, or multiple responses for each request.local_cache
Implementation of a local in-memory cache for Triton Inference Server's TRITONCACHE APIsquare_backend
Simple Triton backend used for testing.Love Open Source and this site? Check out how you can help us