openai-evals 开源项目推荐（9 个）

uptrain-ai /uptrain

UpTrain是一个开源统一平台，用于评估和改进生成性人工智能应用程序。我们为20多个预配置检查（涵盖语言、代码、嵌入用例）提供评分，对失败案例执行根本原因分析并提供有关如何解决它们的见解。

thrivewithai /langchain-fixie-marvin

We compared LangChain, Fixie, and Marvin

plwslpld-arch /eval-harness-internals

面向开发者的中文 Eval Harness 源码教材：解析 lm-evaluation-harness、Inspect AI、OpenAI Evals、Promptfoo、DeepEval 与 Harbor，覆盖任务、运行、评分、统计与发布门禁。

antdragiotis /LLMs-benchmarking-for-SCA-Reg

This project employs LLaMA-Index and OpenAI-Evals frameworks to assess the performance of Large Language Models (LLMs) in accurately answering multiple-choice questions related to the European Regulation on Strong Customer Authentication (SCA) for electronic payments.

sohomx /simtest

SimTest runs deterministic fuzz tests for multi-step LLM agents and fails the PR if schema, policy or cost budgets break.

JayDS22 /openai-evals-enterprise-pack

20 enterprise scenarios for the openai/evals registry format + 3 novel evals the public corpus lacks: tool-use trajectory, streaming-cancel JSON validity, PII leak through tool arguments.

evalsreview /openai-evals-migration

Convert exported OpenAI hosted Evals into promptfoo configs before the 30 Nov 2026 shutdown.

ajaywanekar /evals2promptfoo

Move OpenAI Evals (shutting down Nov 30, 2026) to runnable promptfoo suites: export + convert, grader by grader.

getappai /vf_cot_reasoning.yaml

Evaluation for reasoning consistency under constrained chain-of-thought (VF-CoT). Detects silent correction, decorative CoT, and off channel reasoning.