mechanistic-interpretability 开源项目推荐（1,145 个）

lyramakesmusic /neuron-explorer

Steering-first neuron explorer for Qwen3.5-4B: push every MLP neuron, keep every rollout, browse the results

chrishayuk /larql

Run, serve, query and edit AI models — a Rust model runtime built on VINDEX3, an executable, inspectable model format.

stanfordnlp /pyvene

Stanford NLP Python library for understanding and improving PyTorch models via interventions

johnmarktaylor91 /torchlens

Inspect, visualize and intervene on any operation in any neural network. Capture every activation and gradient, the full computational graph and 800+ metadata fields without changing model code. Supports PyTorch, JAX, TensorFlow, MLX, PaddlePaddle and tinygrad. Validated on 6400+ models. Built for researchers and their AI agents.

zengxiao-he /tessera

从教师到瓦片-从头开始的LLM蒸馏和服务引擎：自定义Triton/CUDA内核、FSDP蒸馏、页面KV连续收件箱、推测性解码、Rust网关、JAX Oracle和可解释性工具。

google-deepmind /detectorscope

Build AI text detectors whose decisions are mechanistically grounded and human-auditable (EMNLP 2026)

ruizheliUOA /Awesome-Interpretability-in-Large-Language-Models

This repository collects all relevant resources about interpretability in LLMs

lordx64 /phantom-kv

Refusal removal as a loadable KV-cache graft — zero weight modification, no refusal-direction projection, fully reversible.

ninjahawk /Subtext

To know what models don't say out loud.

Extraltodeus /J-Wash

Jacobian-Brainwash : A manual alignment tool for large language models built on Anthropic's Jacobian Lens. Results are exportable.

OpenMOSS /Llamascopium

A framework for training, analyzing, and visualizing sparse autoencoders and related interpretability methods

itsqyh /Awesome-LMMs-Mechanistic-Interpretability

A curated collection of resources focused on the Mechanistic Interpretability (MI) of Large Multimodal Models (LMMs). This repository aggregates surveys, blog posts, and research papers that explore how LMMs represent, transform, and align multimodal information internally.

stanfordnlp /axbench

Stanford NLP Python library for benchmarking the utility of LLM interpretability methods

MirroS-Lab /S-Space

S-Space: Exploring Spatial Workspace in Multimodal Models

AI-in-Transportation-Lab /awesome-mechanistic-interpretability

A carefully curated collection of high-quality libraries, projects, tutorials, research papers, and other essential resources focused on Mechanistic Interpretability, a growing subfield in machine learning interpretability research that aims to reverse-engineer neural networks into understandable computational components.

steering-vectors /steering-vectors

Steering vectors for transformer language models in Pytorch / Huggingface

canivel /dynolab

Dyno Lab — an AI safety, alignment and interpretability research workbench for Apple Silicon. Explore activations, probes, SAEs and interventions with local inference, a Python SDK, APIs and MCP.

MadryLab /modelcomponents

Decomposing and Editing Predictions by Modeling Model Computation

AndrewAltimit /template-repo

Agent orchestration & security template featuring MCP tool building, agent2agent workflows, mechanistic interpretability on sleeper agents, and agent integration via CLI wrappers

ndif-team /nnterp

Unified access to Large Language Model modules using NNsight