Cost-efficient and pluggable Infrastructure components for GenAI inference

Community maintained hardware plugin for vLLM on Spyre

Community maintained hardware plugin for vLLM on Ascend

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

A high-throughput and memory-efficient inference and serving engine for LLMs

Community maintained hardware plugin for vLLM on Intel Gaudi

Common recipes to run vLLM

This repo hosts code for vLLM CI & Performance Benchmark infrastructure.

vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs