In BMCook, we implement four representative compression methods, including quantization, pruning, distillation, and MoEfication. Developers can easily combine ...

Task-agnostic compression can provide an effi- cient and versatile big model for both prompt- ing and delta tuning, leading to a more gen- eral impact than task ...

The compression methods used in our experiments include 8bit quantization, structured pruning, unstructured pruning, and MoEfication.

This document introduces BMCook, a task-agnostic compression toolkit for large language models (LLMs) with billions of parameters.

BMCook: A task-agnostic compression toolkit for big models. Z Zhang, B Gong ... BMInf: An Efficient Toolkit for Big Model Inference and Tuning. X Han, G ...

To address the computation bottleneck encountered in deploying big models in real-world scenarios, we introduce an open-source toolkit for big model inference ...

... task-specific compression. Hence, we introduce a task-agnostic compression toolkit BMCook for big models. In BMCook, we implement four representative ...

Aug 24, 2023 · We introduce OpenBMB, an open-source suite of big models, to break the barriers of computation and expertise of big model applications.

Tools. BMCook: Model Compression for Big Models [Code]. llama.cpp: Inference of LLaMA model in pure C/C++ [Code]. LangChain: Building applications with LLMs ...

Nov 15, 2023 · Bmcook: A task- · agnostic compression toolkit for big models. In Pro- ceedings of EMNLP Demonstration, pages 396–405. Zhengyan Zhang, Yankai ...