A high-throughput and memory-efficient inference and serving engine for LLMs

OpenCompass is an LLM evaluation platform, supporting a wide range of models (LLaMA, LLaMa2, ChatGLM2, ChatGPT, Claude, etc) over 50+ datasets.

LLM inference in C/C++

An extended commonmark compliant parser, with bridges to docutils/sphinx

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

Distribute and run LLMs with a single file.

MMDeploy is an open-source deep learning model deployment toolset

OpenMMLab Detection Toolbox and Benchmark

InternLM has open-sourced a 7 billion parameter base model, a chat model tailored for practical scenarios and the training system.

This is the official code for MobileSAM project that makes SAM lightweight for mobile applications and beyond!

Multimodal-GPT

NVIDIA® TensorRT™, an SDK for high-performance deep learning inference, includes a deep learning inference optimizer and runtime that delivers low latency and high throughput for inference applicat…

PPL Quantization Tool (PPQ) is a powerful offline neural network quantization tool.

OpenMMLab Model Compression Toolbox and Benchmark.

OpenMMLab YOLO series toolbox and benchmark

An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.

Simplify your onnx model

NART = NART is not A RunTime, a deep learning inference framework.

Efficient AI Backbones including GhostNet, TNT and MLP, developed by Huawei Noah's Ark Lab.

Open standard for machine learning interoperability

Model Quantization Benchmark

OpenMMLab Image Classification Toolbox and Benchmark

OpenMMLab Computer Vision Foundation

A primitive library for neural network

X-modaler is a versatile and high-performance codebase for cross-modal analytics.

OpenMMLab Text Detection, Recognition and Understanding Toolbox

OpenMMLab Pose Estimation Toolbox and Benchmark.