TensorZero is an open-source stack for industrial-grade LLM applications. It unifies an LLM gateway, observability, optimization, evaluation, and experimentation.

⚙️🦀 Build modular and scalable LLM Applications in Rust

The data plane for agents. Arch is a models-native proxy server that handles the plumbing work in AI: agent routing & hand off, guardrails, end-to-end logs and traces, unified access to LLMs from OpenAI, Anthropic, Ollama, etc. Build agents faster, and scale them reliably.

Bionic is an on-premise replacement for ChatGPT, offering the advantages of Generative AI while maintaining strict data confidentiality

AICI: Prompts as (Wasm) Programs

Open-source LLM load balancer and serving platform for self-hosting LLMs at scale 🏓🦙

A reactive runtime for building durable AI agents

Scalable, fast, and disk-friendly vector search in Postgres, the successor of pgvecto.rs.

🧠 Motorhead is a memory and information retrieval server for LLMs.

Fast, streaming indexing, query, and agentic LLM applications in Rust

Govern & Secure your AI

High-scale LLM gateway, written in Rust. OpenTelemetry-based observability included

Simple, Composable, High-Performance, Safe and Web3 Friendly AI Agents and LazAI Gateway for Everyone

🧭🧭 An intelligent load balancer with smart scheduling that unifies diverse LLMs.

Robot VLM and VLA (Vision-Language-Action) inference API helping you manage multimodal prompts, RAG, and location metadata

Burgonet Gateway is an enterprise LLM gateway that provides secure access and compliance controls for AI systems

Prompt-centric framework for developing LLM applications in Rust

[WIP] Sorai is a lightweight, high-performance, and open-source LLM proxy gateway.

A reactive runtime for building durable AI agents