llm-test 开源项目推荐（18 个）

uptrain-ai /uptrain

UpTrain是一个开源统一平台，用于评估和改进生成性人工智能应用程序。我们为20多个预配置检查（涵盖语言、代码、嵌入用例）提供评分，对失败案例执行根本原因分析并提供有关如何解决它们的见解。

georgian-io /LLM-Finetuning-Toolkit

用于微调、消融和单元测试开源LLM的工具包。

PacificAI /langtest

Deliver safe & effective language models

athina-ai /athina-sdk

LLM Testing SDK that helps you write and run tests to monitor your LLM app in production

zgrai-com /llm-api-test

A tool for testing and comparing the performance of different Large Language Model APIs. 一个用于测试和比较不同大语言模型API性能的工具。

snj07 /req-lab

Open-Source Desktop API client built with Kotlin (KMP). HTTP requests, MCP Client, scripting, variables, collections, and Postman import — runs offline. Alternative to Postman/Insomnia.

biological-alignment-benchmarks /Manipulative-Expression-Recognition

MER is a software that identifies and highlights manipulative communication in text from human conversations and AI-generated responses. MER can evaluate LLM responses for manipulative expressions, fostering development of transparency and safety in AI. It also supports manipulation victims by detecting manipulative patterns in human communication.

pyladiesams /eval-llm-based-apps-jan2025

Create an evaluation framework for your LLM based app. Incorporate it into your test suite. Lay the monitoring foundation.

aralyekta /prompttester

Test, compare, and optimize your AI prompts in minutes

Coldwave96 /LLM-Sec-Evaluation

Scripts for evaluating LLM security abilities.

prompt-foundry /typescript-sdk

The prompt engineering, prompt management, and prompt evaluation tool for TypeScript, JavaScript, and NodeJS.

lalitkpal /VerifyAI

VerifyAI is a verification harness for testing, auditing, and evaluating responses from AI assistants and coding agents.

prompt-foundry /go-sdk

The prompt engineering, prompt management, and prompt evaluation tool for Go.

awesome-software /nlptest

Deliver safe & effective language models

thatoldfarm /logos-infinitum-artifact

A comprehensive corpus of interconnected texts and protocols designed as a conceptual stress-test for advanced AI.

leoli-dev /llm-test-2026-10-20-game-tank-battle

3D tank maze game built by local Qwen3.8-27B 8bit through MLTF and pi; four-round supervised experiment

qbxlvnf11 /llm-test-page

LLM (GPT, Gemini etc.) Test Page: Chat, Parameters, GCP Cloud Run, Inference Time/Token/Cost Calculation etc.

awesome-software /promptfoo

Test your prompts. Evaluate and compare LLM outputs, catch regressions, and improve prompt quality.