This repository provides the Kubernetes infrastructure components, Helm charts, and operational tooling for deploying llm-d - a Kubernetes-native high-performance distributed LLM inference framework.

Infrastructure Components:

• Helm charts for deploying llm-d gateway infrastructure
• Kubernetes Gateway API configurations with support for Istio, kGateway, and GKE
• Service mesh integration and traffic management policies

Operational Tooling:

• Interactive benchmarking and testing utilities
• Smoke testing scripts for deployment validation
• Development helpers and automation tools

• Kubernetes 1.28+ cluster
• Helm 3.10+
• Gateway API v1.3.0+ installed
• Gateway controller (Istio, kGateway, or GKE) deployed in your cluster

# Add the Helm repository
helm repo add llm-d-infra https://llm-d-incubation.github.io/llm-d-infra/
helm repo update

# Install the infrastructure components
helm install my-llm-d-infra llm-d-infra/llm-d-infra

For detailed configuration options, see the Helm chart documentation.

Note: All quickstart guides and user documentation have moved to the main llm-d repository:

• Deployment Guides:

This repository contains:

• Helm Chart Documentation
• Development Guide
• Interactive Testing Tools

1. Issues and Features: Report issues or request features in the main llm-d repository
2. Infrastructure Changes: Submit pull requests to this repository for:
   • Helm chart improvements
   • Operational tooling enhancements
   • Infrastructure configuration updates
3. Development Setup: See development documentation

See CODEOWNERS for component-specific maintainers.

• Helm Charts: Available via the Helm repository: helm repo add llm-d-infra https://llm-d-incubation.github.io/llm-d-infra/

• Release Notes: GitHub releases

This project is licensed under the Apache License 2.0. See the LICENSE file for details.