llm-benchmark 开源项目推荐（514 个）

HowieHwong /TrustLLM

[ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for local models, compatible APIs and AI agent orchestration.

facebookresearch /swe-sweep

How many bugs can LMs find & fix in large codebases?

QuesmaOrg /awesome-ai-tokenomics

A curated list on AI token economics: what tokens cost, where they get wasted, and how to cut the bill. Tools, benchmarks, papers, and copy-paste configs for the token economy of LLMs and coding agents.

thetahealth /mirobody-env

HAEnv — synthetic longitudinal patients with code-derived gold, for evaluating health agents over time

phuryn /bug-hunt-bench

Bug Hunt Bench: 105 real bugs in two production repos, frontier coding models (GPT-6, Claude, Grok, Gemini, DeepSeek...) find and fix them in their own CLI, graded blind. Live leaderboard + every receipt.

TIGER-AI-Lab /WebsiteBench

Can AI agents rebuild websites through browser exploration?

suncityldp /zx-bench

A reproducible local LLM benchmarking platform with versioned datasets, deterministic scoring, and auditable reports.

StudioPlatforms /aistupidmeter-web

AI Stupid Level — independent, continuous LLM benchmarks (coding, reasoning, tool use), drift detection and a benchmark-driven Smart Router. Web app source; methodology public, test tasks private.

composo-ai /DishwasherBench

A dishwasher-loading game on a physical simulation of one real machine, built to benchmark language models and playable in a browser. The game inside is called Spotless.

DeepTempo /socbench

An Open Harness and Benchmark for AI in Cybersecurity Operations.

blackwell-systems /gcf

The AI-native wire format for structured data. 100% comprehension on every frontier model. 50-92% fewer tokens than JSON. 43B+ lossless round-trips across 17 formats. Spec v3.5.1 Stable.

genlab-1c /prism

Открытый бенчмарк LLM: какая нейросеть лучше пишет код 1С:Предприятие (BSL). Объективная оценка LLM по методике SMOP с реальным исполнением в 1С — Claude, GPT, Gemini, DeepSeek, YandexGPT, GigaChat.

enescingoz /mac-llm-bench

Community benchmark database for running LLMs on Apple Silicon Macs

alexshpunt /explicit-edit-benchmark

Benchmark coding agents and agent harnesses on 226 deterministic file-editing tasks with byte-exact verification and a public leaderboard.

TsinghuaC3I /AdsQA

[ICCV 2025] AdsQA: Towards Advertisement Video Understanding Arxiv: https://arxiv.org/abs/2509.08621

HalachaBench /halacha-bench-results

HalachaBench early results: AI models on real semicha exam questions (provisional)

weiber2002 /ICRTL-Benchmark

ICRTL Benchmark: Industrial-level RTL design challenges for evaluating PPA optimization, code generation, and LLM applications in EDA.

yyh-001 /llm-value-rankings

Daily LLM value rankings - compare 300+ models by intelligence, speed and price. OpenRouter + Artificial Analysis. 大模型性价比排行榜

hyeonsangjeon /gdpval-realworks

Open-source benchmark for evaluating LLMs on 220 real professional tasks across 9 sectors and 44 occupations. Reproducible experiments, artifact validation, grading, and a live evidence dashboard.

NEDONION /evalhub

An LLM Benchmark evaluation platform for automated model testing and analysis. 面向大语言模型的 Benchmark 自动化评测平台，支持模型测试与结果分析