This repository provides the Kubernetes infrastructure components, Helm charts, and operational tooling for deploying llm-d - a Kubernetes-native high-performance distributed LLM inference framework.
Infrastructure Components:
- Helm charts for deploying llm-d gateway infrastructure
- Kubernetes Gateway API configurations with support for Istio, kGateway, and GKE
- Service mesh integration and traffic management policies
Operational Tooling:
- Interactive benchmarking and testing utilities
- Smoke testing scripts for deployment validation
- Development helpers and automation tools
- Kubernetes 1.28+ cluster
- Helm 3.10+
- Gateway API v1.3.0+ installed
- Gateway controller (Istio, kGateway, or GKE) deployed in your cluster
# Add the Helm repository
helm repo add llm-d-infra https://llm-d-incubation.github.io/llm-d-infra/
helm repo update
# Install the infrastructure components
helm install my-llm-d-infra llm-d-infra/llm-d-infra
For detailed configuration options, see the Helm chart documentation.
Note: All quickstart guides and user documentation have moved to the main llm-d repository:
This repository contains:
- Issues and Features: Report issues or request features in the main llm-d repository
- Infrastructure Changes: Submit pull requests to this repository for:
- Helm chart improvements
- Operational tooling enhancements
- Infrastructure configuration updates
- Development Setup: See development documentation
See CODEOWNERS for component-specific maintainers.
-
Helm Charts: Available via the Helm repository:
helm repo add llm-d-infra https://llm-d-incubation.github.io/llm-d-infra/
-
Release Notes: GitHub releases
This project is licensed under the Apache License 2.0. See the LICENSE file for details.