Foundation Model Inference (@FMInference)
Guthub

FMInference

Twitter logo Share LinkedIn logo Share Facebook logo Share
  • Stars
    star
    7,170
  • Global Org. Rank 3,272 (Top 2 %)
  • Followers 90
  • Registered over 1 year ago
  • Most used languages
    Python
         100.0 %

Top repositories

1

FlexiGen

Running large language models on a single GPU for throughput-oriented scenarios.
Python
9,171
star
2

H2O

[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.
Python
339
star
3

DejaVu

Python
260
star