AI Observability & Evaluation

Evaluation and Tracking for LLM Experiments and AI Agents

Laminar - open-source all-in-one platform for engineering AI products. Create data flywheel for your AI app. Traces, Evals, Datasets, Labels. YC S24.

Open source platform for AI Engineering: OpenTelemetry-native LLM Observability, GPU Monitoring, Guardrails, Evaluations, Prompt Management, Vault, Playground. 🚀💻 Integrates with 50+ LLM Providers, VectorDBs, Agent Frameworks and GPUs.

Fiddler Auditor is a tool to evaluate language models.

A comprehensive solution for monitoring your AI models in production

A python library to send data to Arize AI!

Monitor, detect, and analyze security threats and Shadow AI in your LLM applications with comprehensive analytics and real-time alerting.

AI runtime inventory: discover shadow AI, trace LLM calls

A report generator library for the ML models deployed on the Fiddler AI Observability platform

Search for a holiday and get destination advice from an LLM. Observability by Dynatrace.

Java client to interact with Arize API

This repo hosts a chatbot that runs in a docker container to demo Okahu AI Observability Cloud

This repo hosts a chatbot that runs in Github Codespaces to demo Okahu AI Observability Cloud with OpenAI

AI Chat Watch (AICW) - free open-source tool for GEO marketers that track what & how AI mentions brands, products, companies.

Example projects for Arthur Model Monitoring Platform

Official Python Library to monitor your LLM Application with Doku

Code used for the demo at AI Day Bhubaneswar 2025