trustworthy-ai - 技术专题

Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams

🐢 Open-Source Evaluation & Testing library for LLM Agents

[ACL 2024] An Easy-to-use Knowledge Editing Framework for LLMs.

Babysitter enforces obedience on agentic workforces and enables them to manage extremely complex tasks and workflows through deterministic, hallucination-free self-orchestration

[EMNLP 2024 Demo] MarkLLM: An Open-Source Toolkit for LLM Watermarking

The open-sourced Python toolbox for backdoor attacks and defenses.

[ICML 2024] TrustLLM: Trustworthiness in Large Language Models

Deliver safe & effective language models

🔥🔥🔥[AAAI 2026 Oral] Official Implementation of Robust-R1: Degradation-Aware Reasoning for Robust Visual Understanding

Proof of thought : LLM-based reasoning using Z3 theorem proving with multiple backend support (SMT2 and JSON DSL)

The testing platform for AI teams. Bring engineers, PMs, and domain experts together to generate tests, simulate (adversarial) conversations, and trace every failure to its root cause.

Moonshot - A simple and modular tool to evaluate and red-team any LLM application.

MarkDiffusion: An Open-Source Toolkit for Generative Watermarking of Latent Diffusion Models

[USENIX Security 2025] PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models

The AI Incident Database seeks to identify, define, and catalog artificial intelligence incidents.

🚀 A fast safe reinforcement learning library in PyTorch

[NeurIPS-2023] Annual Conference on Neural Information Processing Systems

A privacy-first AI agent that sanitizes your prompts locally with a local LLM before forwarding to remote LLM APIs.

A comprehensive toolbox for model inversion attacks and defenses, which is easy to get started.

[NeurIPS'24] "Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration"