Top 80 interpretability open source projects

Class activation maps for your PyTorch models (CAM, Grad-CAM, Grad-CAM++, Smooth Grad-CAM++, Score-CAM, SS-CAM, IS-CAM, XGrad-CAM)

💭 Aspect-Based-Sentiment-Analysis: Transformer & Explainable ML (TensorFlow)

Model interpretability and understanding for PyTorch

Explainable AI framework for data scientists. Explain & debug any blackbox machine learning model with a single line of code.

A collection of research materials on explainable AI/ML

Interpretable ML package 🔍 for concise, transparent, and accurate predictive modeling (sklearn-compatible).

A Python package implementing a new machine learning model for text classification with visualization tools for Explainable AI

A curated list of awesome machine learning interpretability resources.

A game theoretic approach to explain the output of any machine learning model.

📍 Interactive Studio for Explanatory Model Analysis

Layer-wise Relevance Propagation (LRP) for LSTMs

StellarGraph - Machine Learning on Graphs

A curated list of awesome Fairness in AI resources

Pytorch Implementation of recent visual attribution methods for model interpretability

⬛ Python Individual Conditional Expectation Plot Toolbox

Model Agnostics breakDown plots

Awesome Resources for Advanced Computer Vision Topics

Code repository for "Interpretable and Accurate Fine-grained Recognition via Region Grouping", CVPR 2020 (Oral)

Causal Explanation (CXPlain) is a method for explaining the predictions of any machine-learning model.

A collection of tools for reverse engineering neural networks.

🏥 Visualizing Convolutional Networks for MRI-based Diagnosis of Alzheimer’s Disease

A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning

Automatic equation building and curve fitting. Runs on Tensorflow. Built for academia and research.

💡 A curated list of adversarial attacks on model explanations

Text classification models. Used a submodule for other projects.

General Interpretability Package

Contrastive Explanation (Foil Trees), developed at TNO/Utrecht University

Codebase for "Demystifying Black-box Models with Symbolic Metamodels", NeurIPS 2019.

Algorithms for monitoring and explaining machine learning models

[ICCV 2017] Torch code for Grad-CAM

moDel Agnostic Language for Exploration and eXplanation

Interpretability Methods for tf.keras models with Tensorflow 2.x

A collection of anomaly detection methods (iid/point-based, graph and time series) including active learning for anomaly detection/discovery, bayesian rule-mining, description for diversity/explanation/interpretability. Analysis of incorporating label feedback with ensemble and tree-based detectors. Includes adversarial attacks with Graph Convolutional Network.

Federated Learning Library: https://fedml.ai

XAI - An eXplainability toolbox for machine learning

Visualization toolkit for neural networks in PyTorch! Demo -->

Interesting resources related to XAI (Explainable Artificial Intelligence)

Examples of techniques for training interpretable ML models, explaining ML models, and debugging ML models for accuracy, discrimination, and security.

Public facing deeplift repo

Code for the TCAV ML interpretability project

A collection of infrastructure and tools for research in neural network interpretability.

H2O.ai Machine Learning Interpretability Resources

Making decision trees competitive with neural networks on CIFAR10, CIFAR100, TinyImagenet200, Imagenet

深度学习近年来关于神经网络模型解释性的相关高引用/顶会论文(附带代码)

Fit interpretable models. Explain blackbox machine learning.

Human-explainable AI.

Many Class Activation Map methods implemented in Pytorch for CNNs and Vision Transformers. Including Grad-CAM, Grad-CAM++, Score-CAM, Ablation-CAM and XGrad-CAM

Sample use case for Xavier AI in Healthcare conference: https://www.xavierhealth.org/ai-summit-day2/

A lightweight implementation of removal-based explanations for ML models.

Code for SPINE - Sparse Interpretable Neural Embeddings. Jhamtani H.*, Pruthi D.*, Subramanian A.*, Berg-Kirkpatrick T., Hovy E. AAAI 2018

Towards Understanding Deep Learning Representations via Interactive Experimentation

A library for finding knowledge neurons in pretrained transformer models.

[ECCV 2018] code for Choose Your Neuron: Incorporating Domain Knowledge Through Neuron Importance

🏔️ Summit: Scaling Deep Learning Interpretability by Visualizing Activation and Attribution Summarizations

For calculating global feature importance using Shapley values.

A collection of state-of-the-art algorithms for the training, serving and interpretation of Decision Forest models.

Zennit is a high-level framework in Python using PyTorch for explaining/exploring neural networks using attribution methods like LRP.

ProtoTrees: Neural Prototype Trees for Interpretable Fine-grained Image Recognition, published at CVPR2021

official implementation of "Visualization of Convolutional Neural Networks for Monocular Depth Estimation"

Code for using CDEP from the paper "Interpretations are useful: penalizing explanations to align neural networks with prior knowledge" https://arxiv.org/abs/1909.13584