Skip to content

Pinned Loading

  1. vllm vllm Public

    A high-throughput and memory-efficient inference and serving engine for LLMs

    Python 53k 8.9k

  2. llm-compressor llm-compressor Public

    Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

    Python 1.7k 182

  3. recipes recipes Public

    Common recipes to run vLLM

    33 3

Repositories

Showing 10 of 20 repositories