When LLM Agents Meet Reinforcement Learning
AgentsMeetRL is an awesome list that summarizes open-source repositories for training LLM Agents using reinforcement learning:
- 🤖 The criteria for identifying an agent project are that it must have at least one of the following: multi-turn interactions or tool use (so TIR projects, Tool-Integrated Reasoning, are considered in this repo).
- ⚠️ This project is based on code analysis from open-source repositories using LLM coding agents, which may contain unfaithful cases. Although manually reviewed, there may still be omissions. If you find any errors, please don’t hesitate to let us know immediately through issues or PRs - we warmly welcome them!
- 🚀 We particularly focus on the reinforcement learning frameworks, RL algorithms, rewards, and environments that projects depend on, for everyone’s reference on how these excellent open-source projects make their technical choices. See [Click to view technical details] under each table.
- 📅 Last updated: 2026-10-10
- 🤗 Feel free to submit your own projects anytime - we welcome contributions!
- 📚 If you find this repository helpful for your research, please cite it via the “Cite this repository” button on the right sidebar.
Taxonomy:
- Base Framework: General-purpose RL training frameworks for LLM agents (e.g., veRL, OpenRLHF, trl)
- General/MultiTask: Agent systems trained/evaluated across multiple tasks or environments
- Search & RAG: Search-augmented reasoning agents that use retrieval tools to enhance LLM reasoning
- Web & GUI: Agents that interact with web browsers, mobile/desktop GUIs, or operating systems
- Tool-Use: Agents trained to invoke external tools (APIs, code executors, MCP, etc.)
- Code & SWE: Software engineering and code generation agents
- Reasoning: Reasoning agents with tool-integrated or multi-turn reasoning (math, QA, visual)
- Multi-Agent RL: Multi-agent collaboration, negotiation, or credit assignment via RL
- Memory: Agents that learn to manage, retrieve, or evolve memory
- Embodied: Agents operating in embodied/physical simulation environments
- Domain-Specific: RL agents for specialized domains (medical, OS tuning, etc.)
- Reward & Training: Process/outcome reward models and training methodologies for agents
- Safety: RL for agent safety alignment, adversarial red-teaming, and jailbreak defense/attack
- VLM Agent: Vision-language model agents trained with RL for multimodal interaction
- Self-Evolution: Agents that self-evolve via RL feedback loops (⚠️ definition still evolving in the community)
- Environment: Benchmarks, gyms, and sandbox environments for agent training/evaluation
Some Enumeration:
- Enumeration for Reward Type:
- External Verifier: e.g., a compiler or math solver
- Rule-Based: e.g., a LaTeX parser with exact match scoring
- Model-Based: e.g., a trained verifier LLM or reward LLM
- Custom
Updates
- 📢 2026-10 Update: Added 3 new repositories (Base Framework +1 [LoongSage, Baidu’s production-scale async agentic-RL framework on Megatron-Core + SGLang], Environment +2 [MIMESIS, a trained user simulator used as a multi-turn RL environment; plus catch-up of Endless Terminals, procedurally generated terminal-agent RL environments]), on top of the community-contributed DRACO (Reward & Training) merged in September. Each repo was opened and confirmed to contain real RL-training code. SAPO was added to Under Review (repo is README-only so far); many Sep–Oct papers (e.g. RLVR for Small Search Agents, HazardAuditor, T1, Qwen-Planner-Agent, ARISE-RL, TRCA) were left out because their code is not yet public. None of the previous Under Review items have released RL training code yet.
- 📢 2026-08 Update: Added 23 new repositories across 9 categories (Environment +8 [Echoverse/PAST-Bench/PatientAgentBench/LegalWorld/Evo-Bench/DocOps/ScrambleToolBench/DigiWorld], Search & RAG +5 [EviSD/GTA-RAG/LAPO, plus catch-up of GrepSeek/PyRAG], Base Framework +3 [Molt, AReno, and catch-up of Microsoft Orchard], Self-Evolution +2 [AgentOPSD/BaT], Code & SWE +1 [Lego-RL, harness-native RL inside Claude Code/OpenHands/OpenCode], Reward & Training +1 [Agent-G²], VLM Agent +1 [InSight-doc], Memory +1 [MemPrism], Tool-Use +1 [MUA-RL, promoted from Under Review — its code had in fact been public since 2025.8]). Every repo was opened and confirmed to contain real RL-training (or executable-environment) code. Papers whose code is still unreleased were left out (Qwen-UI-Agent, Qwen-CUA, UI-Mate, SearchMaster, RoMeRL, Agon, SINKFLEX-RL, GRASP, MAVEN, EviBack, ChemWorld) — see Under Review. Notably no qualifying new Safety, Embodied, or Multi-Agent RL repos appeared this window: that crop was uniformly SFT-only, inference-only, or code-withheld.
- 📢 2026-07 Update: Added 13 new repositories from late-Jun–Jul 2026 across 8 categories (Self-Evolution +3 [SEED/OPID/UCOB, the on-policy-distillation-for-agentic-RL line], VLM Agent +2 [VTS/VSeek, long-video search agents], Tool-Use +2 [Tool-RL-Box; plus catch-up of AWorld-RL], Environment +2 [SETA terminal envs, OpenAgent tool-generalization sandbox], Web & GUI +1 [SCALE-CUA], Embodied +1 [REAL], Memory +1 [Supersede], Domain-Specific +1 [FaithMed]). Every entry was verified by opening the repo and confirming real RL-training (or environment) code — papers whose code is not yet released (EvoCUA-1.5, DeepSearch-World, CompactionRL, GUICrafter, VideoSearcher, Xiaomi-GUI-0) were deliberately left out.
- 📢 2026-06 Update: Added 43 new repositories across 11 categories (VLM Agent +8, Search & RAG +7, Environment +6, Reward & Training +4, Base Framework/Tool-Use/Self-Evolution/Embodied +3 each, Web & GUI/Code & SWE/Domain-Specific +2 each). New since the last update: Harness-1, FastContext, OpenWebRL, Polar, AgentJet, HarnessX, APPO, SPADER, DeepRubric, Embodied-R1.5, SIRI; plus catch-up of earlier-2026 misses (Vision-DeepResearch, ARM-Thinker, PyVision-RL, Gen-Searcher, DataMind, Tool-R0, Agent World Model, VisGym, Gym-Anything, ChemCraft, OpAgent, etc.).
- 📢 2026-05 Update: Added 17 new repositories from Apr–May 2026 across 11 categories (notably General/MultiTask +4 [SkillZero/T²PO/SDAR/StraTA, mostly ZJU-REAL & related agentic RL methods], VLM Agent +3 [MTA-Agent/ParaVT/OpenSearch-VL, multimodal deep search & video tool use], Web & GUI +2 [ClawGUI/ToolCUA]). Moved CoEvolve to “Under Review” (code not yet released).
- 📢 2026-04 Update: Added 67 new repositories covering Apr 2025 – Apr 2026 across nearly every category (notably VLM Agent +9, Search & RAG +10, Web & GUI +7, Tool-Use +7). Also reclassified SkyRL (→ General) and SPIRAL (→ Multi-Agent), and updated the VAGEN entry to its NeurIPS’25 upstream repo.
- 📢 2026-03 Update: Restructured taxonomy from 12 to 16 categories (added Multi-Agent RL, Reward & Training, Safety, VLM Agent, Self-Evolution, Domain-Specific; merged GUI into Web & GUI; retired TextGame/Biomedical). Added ~70 new repositories covering Sep 2025 – Mar 2026, growing the total from ~134 to 205.
🤖 Use as a Claude Code Skill
This list is also packaged as a Claude Code Skill — agents-meet-rl — that turns the corpus into an on-demand assistant for agentic-RL training, evaluation, and experiment design: reward not moving, KL / entropy / length blow-ups, GRPO / PPO / DAPO knobs, retokenization drift, tool-call parse failures, long-horizon credit assignment, LLM-judge inconsistency, benchmark contamination, and framework / benchmark / algorithm selection — each answer anchored to specific papers and repos from this list. Backed by a machine-readable corpus of 411 projects (snapshot 2026-10-10). Once installed, Claude Code auto-invokes it whenever your question matches.
The skill also includes an engineering handbook
covering trajectory contracts, token/mask alignment, multi-turn objectives,
async correction, environments, rewards, distillation, and long-horizon evaluation.
Versioned implementation cards
trace slime, veRL, and AReaL source snapshots; offline checks
help inspect exported trajectories. Engineering review: 2026-11-06; these
source inspections are not claims of training or quality validation.
Install as a plugin (recommended):
/plugin marketplace add thinkwee/claude-plugins
/plugin install agents-meet-rl@thinkwee
Or install manually:
git clone https://github.com/thinkwee/AgentsMeetRL
cp -r AgentsMeetRL/skills/agents-meet-rl ~/.claude/skills/
Then just ask, e.g. “my GRPO search agent’s reward is flat but eval keeps dropping” or “which RL framework should I pick for a multi-turn tool-use agent?” — the skill routes your symptom to fixes grounded in this corpus.
🔧 Base Framework
📋 Click to view technical details
| Github Repo |
RL Algorithm |
Single/Multi Agent |
Outcome/Process Reward |
Single/Multi Turn |
Task |
Reward Type |
Tool usage |
| LoongSage |
Critic-free PG (GRPO-family) + online distillation (PG/GKD-style), fully async |
Single |
Outcome |
Multi |
Production-scale agentic RL on MoE models (Megatron-Core + SGLang; math, agentic code repair) |
Rule/External |
Yes (AgentFlow: tool output recognition, sandbox lifecycle) |
| Libra |
Async GRPO |
Single |
Outcome |
Multi |
Agentic RL post-training with resource-aware training and rollout |
All (Custom/External/Rule) |
Yes (search, code execution, validation tools) |
| Orchard |
Online RL (vendored slime) |
Single |
Both |
Multi |
Harness-native RL (browser/computer-use/SWE) |
Model/Rule |
Yes (live browser, containers, agent harnesses) |
| AReno |
GSPO/GRPO (+SFT/DPO) |
Single |
Outcome |
Multi |
Single-node RL post-training (Math/SWE-style/Games) |
Custom |
Yes (inspect_tree/read_file/rg/apply_patch/run_command) |
| Roger Federated |
QLoRA REINFORCE++ (federated, secure aggregation) |
Single |
Both |
Multi |
On-device RL on the agent chats the user’s own llama.cpp/vLLM server already handles |
Model/External (self-evaluation, user reactions, tool results) |
Yes (whatever tools the user’s client calls) |
| Molt |
REINFORCE/RLOO/GRPO/Dr.GRPO/GAE + On-policy Distillation |
Single |
Outcome |
Multi |
Domain-agnostic agentic RL (Math/Geometry/Chat) |
Custom |
Yes (Python exec, HTTP, VLM tools) |
| AgentJet |
GRPO/PPO (swarm, multi-dim reward) |
Both |
Both |
Multi |
Swarm agentic RL (heterogeneous multi-agent, multi-task) |
All (Custom/External/Rule) |
Yes (tool calls, agent frameworks) |
| HarnessX |
GRPO/PPO (slime/verl recipes) |
Single |
Outcome |
Multi |
Composable agent-harness foundry (ALFWorld/GAIA/WebShop/SWE-bench) |
External + Custom |
Yes (harness orchestrates tools/memory) |
| Dressage |
GRPO |
Both |
Outcome |
Multi |
Agentic RL for any agent and sandbox (SWE-Gym/ALFWorld/HotpotQA) |
External/Rule |
Yes (whitebox: code/shell/file/retrieval; blackbox: opencode/openclaw/claude_code/codex) |
| Polar |
GRPO |
Both |
Outcome |
Multi |
Agentic RL on any harness (SWE-Bench/SWE-Gym) |
External Verifier |
Yes (real agent harnesses: shell/Codex/Claude Code) |
| uni-agent |
GRPO/GSPO (partial rollout, fully-async) |
Single |
Outcome |
Multi |
SWE-Bench/Search/General Agent (1000+ concurrent) |
All |
Yes (unified model/tool/env abstractions) |
| VeRL-Omni |
FlowGRPO/DanceGRPO/Diffusion DPO |
Single |
Outcome |
Single |
Multimodal generation RL (image/video/omni) |
Model/External |
No |
| OpenClaw-RL |
GRPO/OPD |
Both |
Both |
Multi |
Terminal/GUI/SWE/Tool-call |
Model/External |
Yes |
| Claw-R1 |
Generic RL Framework |
Multi |
Both |
Multi |
General Agent |
All |
Yes (Framework-agnostic) |
| Open-AgentRL |
GRPO-TCR |
Single |
Both |
Multi |
Reasoning/GUI/Coding |
Model (PRM) |
Yes (SandboxFusion) |
| NeMo-RL |
GRPO/DAPO/GDPO/DPO |
Single |
Outcome |
Multi |
Math/Reasoning/Code |
Rule/External |
No |
| RLinf |
PPO/GRPO/DAPO/SAC/REINFORCE++/CrossQ/RLPD |
Both |
Both |
Multi |
Robotics/Math/Code/QA/VQA |
All (Rule/Model/External) |
Yes |
| siiRL |
PPO/GRPO/CPGD/MARFT |
Multi |
Both |
Multi |
LLM/VLM/LLM-MAS PostTraining |
Model/Rule |
Planned |
| slime |
GRPO/GSPO/REINFORCE++ |
Single |
Both |
Both |
Math/Code |
External Verifier |
Yes |
| agent-lightning |
PPO/Custom/Automatic Prompt Optimization |
Multi |
Outcome |
Multi |
Calculator/SQL |
Model/External/Rule |
Yes |
| AReaL |
PPO |
Both |
Outcome |
Both |
Math/Code |
External |
Yes |
| ROLL |
PPO/GRPO/Reinforce++/TOPR/RAFT++ |
Multi |
Both |
Multi |
Math/QA/Code/Alignment |
All |
Yes |
| MARTI |
PPO/GRPO/REINFORCE++/TTRL |
Multi |
Both |
Multi |
Math |
All |
Yes |
| Tunix |
PPO/GRPO/GSPO-Token/DAPO/Dr.GRPO |
Single |
Outcome |
Multi |
Math/Code/Game |
Rule/External |
Yes |
| RL2 |
Dr. GRPO/PPO/DPO |
Single |
Both |
Both |
QA/Dialogue |
Rule/Model/External |
Yes |
| verifiers |
GRPO |
Multi |
Outcome |
Both |
Reasoning/Math/Code |
All |
Code |
| prime-rl |
GRPO/PPO |
Multi |
Outcome |
Multi |
Math/Code/Search |
Model/External |
Yes |
| oat |
PPO/GRPO |
Single |
Outcome |
Multi |
Math/Alignment |
External |
No |
| veRL |
PPO/GRPO |
Single |
Outcome |
Both |
Math/QA/Reasoning/Search |
All |
Yes |
| OpenRLHF |
PPO/REINFORCE++/GRPO/DPO/IPO/KTO/RLOO |
Multi |
Both |
Both |
Dialogue/Chat/Completion |
Rule/Model/External |
Yes |
| trl |
PPO/GRPO/DPO |
Single |
Both |
Single |
QA |
Custom |
No |
💪 General/MultiTask
📋 Click to view technical details
| Github Repo |
RL Algorithm |
Single/Multi Agent |
Outcome/Process Reward |
Single/Multi Turn |
Task |
Reward Type |
Tool usage |
| T2PO |
T²PO (token+turn uncertainty-guided) |
Single |
Both |
Multi |
WebShop/ALFWorld/SearchQA/Embody/Game |
Rule |
Yes (search, web, embodied) |
| StraTA |
Hierarchical GRPO + Strategic Trajectory Abstraction |
Single |
Outcome |
Multi |
ALFWorld (93.1%)/WebShop (84.2%)/SciWorld (63.5%) |
Rule + Model (self-judge) |
Yes (interactive long-horizon envs) |
| SDAR |
Self-Distilled Agentic RL (GRPO + gated OPSD) |
Single |
Outcome |
Multi |
ALFWorld/WebShop/Search-QA |
Rule |
Yes (interactive envs) |
| SkillZero |
In-Context Agentic RL (GRPO + skill-context curriculum withdrawal) |
Single |
Outcome |
Multi |
ALFWorld/WebShop/Search-QA |
Rule |
Yes (interactive envs + skill library) |
| MetaClaw |
GRPO (LoRA) |
Single |
Process |
Multi |
General Agentic |
Model (PRM) |
Yes (Skill-augmented) |
| SkillRL |
GRPO |
Single |
Outcome |
Multi |
ALFWorld/WebShop/Search |
Rule |
Yes (Web search, actions) |
| LLM-in-Sandbox |
GRPO++ |
Single |
Outcome |
Multi |
Math/Physics/Chemistry/Biomedicine/Long-context/IF/SWE |
Rule |
Yes (Code Sandbox w/ Terminal, File, Internet) |
| youtu-agent |
Training-Free GRPO |
Single |
Outcome |
Multi |
Deep Research/Data Analysis/Tool-use |
Model/External |
Yes (Web search, code, file) |
| DEPO |
KTO + Efficiency Loss |
Single |
Both |
Multi |
Agent (BabyAI/WebShop) |
Rule |
Yes |
| SPEAR |
GRPO/GiGPO + SIL |
Single |
Both |
Multi |
Math/Agent |
Rule/External |
Yes (Search, Sandbox, Browser) |
| DeepAgent |
ToolPO |
Single |
Outcome |
Multi |
ToolBench/ALFWorld/WebShop/GAIA/HLE |
Model |
Yes (16,000+ RapidAPIs) |
| AgentRL |
GRPO/REINFORCE++/RLOO/ReMax/GAE |
Single |
Outcome |
Multi |
Agent Tasks |
External |
Yes |
| AgentGym-RL |
PPO/GRPO/RLOO/REINFORCE++ |
Single |
Outcome |
Multi |
Web/Search/Game/Embodied/Science |
Rule/Model/External |
Yes (Web, Search, Env APIs) |
| Agent_Foundation_Models |
DAPO/PPO |
Single |
Outcome |
Single |
QA/Code/Math |
Rule/External |
Yes |
| Trinity-RFT |
PPO/GRPO |
Single |
Outcome |
Both |
Math/TextGame/Web |
All |
Yes |
| SPA-RL-Agent |
PPO |
Single |
Process |
Multi |
Navigation/Web/TextGame |
Model |
No |
| verl-agent |
PPO/GRPO/GiGPO/DAPO/RLOO/REINFORCE++ |
Multi |
Both |
Multi |
Phone Use/Math/Code/Web/TextGame |
All |
Yes |
| SkyRL |
GRPO/PPO |
Single |
Both |
Multi |
Long-horizon Agents (SWE-Bench/Search/Math/SQL) |
Rule/External/Custom |
Yes |
| VAGEN |
PPO/GRPO (World Modeling RL) |
Single |
Both |
Multi |
Navigation/TextGame/Multimodal |
All |
Yes |
| ART |
GRPO |
Multi |
Both |
Multi |
TextGame |
All |
Yes |
| OpenManus-RL |
PPO/DPO/GRPO |
Multi |
Outcome |
Multi |
TextGame |
All |
Yes |
🔍 Search & RAG Agent
📋 Click to view technical details
| Github Repo |
RL Algorithm |
Single/Multi Agent |
Outcome/Process Reward |
Single/Multi Turn |
Task |
Reward Type |
Tool usage |
| PyRAG |
Curriculum shared-parameter GRPO (LoRA) |
Multi (Decompose/Plan/Answer) |
Both |
Multi |
Multi-hop RAG via executable Python synthesis |
Rule-Based + Execution |
Yes (Python exec + E5 retriever) |
| GrepSeek |
SFT cold-start + GRPO |
Single |
Outcome |
Multi |
Direct corpus interaction (shell/grep, no index) |
Rule-Based (token-F1 x format gate) |
Yes (shell pipelines over raw corpus) |
| LAPO |
GRPO + Leave-One-Turn Attribution |
Single |
Both |
Multi |
Multi-turn search QA (NQ/TriviaQA/HotpotQA/2Wiki) |
Rule-Based (self-generated) |
Yes (retrieval) |
| GTA-RAG |
3-stage GRPO |
Single |
Both |
Multi |
Multi-hop QA over entity-document graph |
Rule-Based (support-doc + EM) |
Yes (graph + dense retrieval) |
| EviSD |
GRPO + Evidence-Conditioned Self-Distillation |
Single |
Outcome |
Multi |
Search-augmented multi-hop QA |
Rule-Based |
Yes (search/retrieval) |
| Harness-1 |
GRPO |
Single |
Outcome |
Multi |
Long-horizon search (web/finance/patents) w/ state-externalizing harness |
External + Rule |
Yes (search/retrieval/rerank) |
| SlimSearcher |
GRPO + Adaptive Reward Gating |
Single |
Outcome |
Multi |
Efficiency-aware deep research (GAIA/BrowseComp/xBench) |
Custom + Rule |
Yes (web search, browse) |
| DeepRubric |
GRPO + rubric rewards |
Single |
Process |
Multi |
Deep research report synthesis (evidence-tree rubric) |
Model + Rule (rubric) |
Yes (search/browse/scholar) |
| SAAS |
RL w/ boundary-aware reward (2-stage curriculum) |
Single |
Outcome |
Multi |
Self-aware agentic search (over-search mitigation, 7 QA sets) |
Rule-Based |
Yes (search) |
| CuSearch |
GRPO + Search-Depth curriculum rollout |
Single |
Outcome |
Multi |
Agentic RAG multi-hop QA |
Rule-Based (EM) |
Yes (retrieval/search) |
| ORBIT |
GRPO |
Single |
Outcome |
Multi |
Verifiable data-gen + RL for web search (Qwen3-4B) |
External + Rule |
Yes (web search) |
| LiteResearcher |
Scalable Agentic RL (curriculum w/ lite virtual world) |
Single |
Outcome |
Multi |
Deep Research (GAIA 71.3% / Xbench-DS 78.0%, 4B SOTA) |
Rule/External |
Yes (local search/browse env, Milvus+PostgreSQL) |
| DR-Venus |
GRPO + IGPO (info-gain turn-level) w/ agentic SFT |
Single |
Both |
Multi |
Edge-scale Deep Research (4B) |
Intrinsic (info-gain) + Rule (format) |
Yes (Search/Browse) |
| MR-Search |
In-context Meta-RL (multi-episode credit) |
Single |
Outcome |
Multi |
Agentic search w/ self-reflection |
Rule-Based |
Yes (search) |
| ProRAG |
GRPO + DGA (dual-granularity advantage) |
Single |
Both |
Multi |
Multi-hop RAG |
Model (PRM via MCTS) |
Yes (Retrieval) |
| O-Researcher |
GRPO + RLAIF |
Multi |
Process |
Multi |
Deep Research (Zhihu-KOL/WideSearch/ELI5) |
Model (LLM-as-Judge) |
Yes (Search/Crawl) |
| Agentic-RAG-R1 |
GRPO |
Single |
Outcome |
Multi |
Knowledge-intensive QA |
Rule/Model |
Yes (Wiki/Doc search) |
| MemSearcher |
Multi-context GRPO |
Single |
Outcome |
Multi |
Search/QA + Memory |
Rule/Model |
Yes (Web search + Memory) |
| DR Tulu |
GRPO + evolving rubrics |
Single |
Outcome |
Multi |
Long-form Deep Research |
Model (rubrics) |
Yes (Search/MCP) |
| IGPO |
GRPO + IGPO (Information Gain turn-level reward) |
Single |
Both |
Multi |
Multi-turn Search Agent (BrowseComp/-ZH) |
Intrinsic (belief Δ) + Outcome |
Yes (Search) |
| ReSeek |
GRPO/PPO |
Single |
Both |
Multi |
QA/Search |
Rule |
Search/JUDGE |
| AutoGraph-R1 |
GRPO (via VeRL) |
Single |
Outcome |
Multi |
KG Construction for QA |
Rule |
Yes (Graph retrieval) |
| WebSeer |
GRPO-style |
Single |
Outcome |
Multi |
Web Search QA (w/ self-reflection) |
Rule/Model |
Yes (Search) |
| HiPRAG |
PPO |
Single |
Process |
Multi |
Efficient Agentic RAG |
Model/Rule |
Yes (Retrieval) |
| Tree-GRPO |
GRPO/Tree-GRPO |
Single |
Outcome |
Multi |
Search |
Rule |
Search |
| DeepResearch |
RL-based |
Single |
Outcome |
Multi |
Deep Research |
Model |
Yes (Search, Browse) |
| DeepDive |
GRPO |
Single |
Outcome |
Multi |
KG-augmented Search |
Rule |
Yes (KG + Search) |
| ASearcher |
PPO/GRPO + Decoupled PPO |
Single |
Outcome |
Multi |
Math/Code/SearchQA |
External/Rule |
Yes |
| SSRL |
GRPO |
Single |
Outcome |
Multi |
Self-Search |
Rule |
Yes (Self-search) |
| Research-Venus |
GRPO |
Single |
Both |
Multi |
Deep Research |
Model (atomic thought) |
Yes (Search) |
| Graph-R1 |
GRPO/REINFORCE++/PPO |
Single |
Outcome |
Multi |
KGQA |
Rule (EM/F1) |
Yes (Graph retrieval) |
| Kimi-Researcher |
REINFORCE |
Single |
Outcome |
Multi |
Research |
Outcome |
Search, Browse, Coding |
| R-Search |
PPO/GRPO |
Single |
Both |
Multi |
QA/Search |
All |
Yes |
| R1-Searcher-plus |
Custom |
Single |
Outcome |
Multi |
Search |
Model |
Search |
| StepSearch |
PPO |
Single |
Process |
Multi |
QA |
Model |
Search |
| AutoRefine |
PPO/GRPO |
Multi |
Both |
Multi |
RAG QA |
Rule |
Search |
| ZeroSearch |
PPO/GRPO/REINFORCE |
Single |
Outcome |
Multi |
QA/Search |
Rule |
Yes |
| ReasonRAG |
DPO + MCTS-based PRM |
Single |
Process |
Multi |
Multi-hop QA |
Model (PRM) |
Yes (Wikipedia search) |
| VRAG |
GRPO |
Single |
Both |
Multi |
Visually-rich RAG |
Rule/Model |
Yes (Visual retrieval) |
| MaskSearch |
DAPO |
Single |
Outcome |
Multi |
RAMP Pretraining + QA |
Rule/Model |
Yes (Search) |
| R3-RAG |
PPO |
Single |
Both |
Multi |
Multi-hop QA |
Rule |
Yes (Retrieval) |
| O2-Searcher |
GRPO |
Single |
Outcome |
Multi |
Open-ended QA |
Rule/Model |
Yes (Search) |
| s3 |
GRPO |
Single |
Outcome |
Multi |
RAG / Medical QA |
Model (Gain-Beyond-RAG) |
Yes (Retrieval) |
| knowledge-r1 |
GRPO |
Single |
Outcome |
Multi |
Knowledge-intensive QA (KB-aware) |
Rule |
Yes (Retrieval) |
| WebThinker |
DPO |
Single |
Outcome |
Multi |
Reasoning/QA/Research |
Model/External |
Web Browsing |
| DeepResearcher |
PPO/GRPO |
Multi |
Outcome |
Multi |
Research |
All |
Yes |
| Search-R1 |
PPO/GRPO |
Single |
Outcome |
Multi |
Search |
All |
Search |
| R1-Searcher |
PPO/DPO |
Single |
Both |
Multi |
Search |
All |
Yes |
| C-3PO |
PPO |
Multi |
Outcome |
Multi |
Search |
Model |
Yes |
| DeepRetrieval |
GRPO |
Single |
Outcome |
Multi |
Query Generation/IR |
Rule |
Yes (Search) |
🌐 Web & GUI Agent
README 内容过长，已截取前 64 KB。完整内容请前往 GitHub 仓库查看。
评论