Evidently is ​​an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

Sister project to OpenLLMetry, but in Typescript. Open-source observability for your LLM application, based on OpenTelemetry

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

High-scale LLM gateway, written in Rust. OpenTelemetry-based observability included

Free MLOps course from DataTalks.Club

LLM application tracing based on OpenTelemetry

A toolkit for evaluating and monitoring AI models in clinical settings

DriftRadar-Vision is an advanced, production-ready drift monitoring and adaptive retraining pipeline for vision models.

Monitoring for AI Applications

statistical tests for drift detection and dataset shift

A python library to send data to Arize AI!

Scalable and production-ready MLOps pipeline using MLflow, Docker, Prometheus & GitHub Actions — featuring automated retraining and model monitoring.

Deepchecks: Tests for Continuous Validation of ML Models & Data. Deepchecks is a holistic open-source solution for all of your AI & ML validation needs, enabling to thoroughly test your data and models from research to production.

Experiments with Model Training, Deployment & Monitoring

A comprehensive toolkit for end-to-end continued pre-training, fine-tuning, monitoring, testing and publishing of language models with MLX-LM

nannyml: post-deployment data science in python

OpenLLM Monitor is a plug-and-play, real-time observability dashboard for monitoring and debugging LLM API calls across OpenAI, Ollama, OpenRouter, and more. Tracks tokens, latency, cost, retries, and lets you replay prompts — fully open-source and self-hostable.

A production-ready ML pipeline for fraud detection built with Apache Airflow, Docker, and FastAPI. Includes modular ETL (bronze → silver → gold), automated model training with Optuna tuning, real-time inference via REST API, and monitoring with MLflow and EvidentlyAI. Designed for scalability, MLOps integration, and real-world deployment.