academic-commercialization-agent

Academic Commercialization Assessment Agent

Paper in, cited commercialization triage out. A multi-agent LLM pipeline turns a research paper or topic into a scorecard for technology readiness, manufacturing, patents and market, with citations to registered sources and the risks spelled out. Its experiments are published with their protocols, failures included.

▶ Read a finished report — no key, no sign-up · Open the application · Run locally · 中文

A real run from 2026-08-14 (CAR-T cell therapy for blood cancers, deepseek-chat), opened from a local copy in the current interface and captured 2026-10-23; no model or search request was made. It predates today’s automated checks, which the panel states; it is not a current-model result.

Try it

The samples are also in examples/: five runs of 2026-11-08 on the current pipeline with deepseek-v4-pro, such as Continuous glucose monitoring for diabetes and the Chinese 特高压直流输电, and the Hyperloop production run of 2026-10-21. Three July reports stay in the folder for the frozen comparisons that read them.

What makes it different

• Evidence first, model second. Code retrieves, validates and freezes the sources; six LLM stages then reason only over that registry, and the weighted total is computed by code, not the model.
• Measured in the open. Protocols, held-out checks and errata live in the repository. A four-agent variant used 54.89% fewer median tokens than the six-agent default, and the first held-out cohort found TRL scores compressed toward 7, under-scoring mature technologies. See Measured results.
• Built for runs that fail halfway. An interrupted run resumes from its last validated checkpoint as a new linked run. Paid requests get receipts, so after a lost reply the result can be recovered for up to 24 hours without paying again.

Built with Python, CrewAI, FastAPI and a build-free JavaScript client. Start here: documentation and repository map · Case study

Want to contribute? Issues labelled good first issue need no API keys: the test suite and browser smokes make no paid requests. Start with CONTRIBUTING.md.

Finding your way around: the application is src/academic_agent/, api/, web/ and ui/; tests are in tests/ and e2e/. The many Python files at the root are benchmark commands and frozen experiment runners whose protocols bind their paths, so they stay put; none is needed to run the app. See the repository map for every directory.

What it does

• Submit a research topic or attach a paper PDF; choose the report language and scoring profile.
• Deterministic code retrieves and validates sources first; six LLM stages then reason over that frozen evidence. Autonomy is limited on purpose.
• The scorecard covers TRL, MRL, patent strength, market accessibility and evidence confidence. The weighted total is computed by code, not the model.
• Follow progress, inspect citations and reliability warnings, export Markdown or PDF, and share a run link.
• In Sources, find a saved source by ID or keyword and read its saved text without a model call. Saved text is not the paper’s full text or verified claim support; see the operating guide.
• Optionally add Decision Context: the report distinguishes exploratory orientation from a specific actor’s decision, and whether a threshold was approved by the decision owner.
• Resume an interrupted run from its last validated checkpoint. The resumed run is a new linked run; the original is never overwritten.
• Paid requests get durable receipts: after a lost reply or a refresh, the accepted run, PDF or resumed run can be recovered for up to 24 hours without paying again.
• With an account, the keys you save are checked with their providers before the first run, the search plan is shown for editing before a run starts, and the composer shows what is left of the day’s allowance. Your history is searchable in the page; a finished run offers Run again, Copy link and Compare, which puts two runs’ scorecards side by side with the difference between them (how).
• When a run fails, the page says why in words for whoever can act on it: a provider refusing the deployment’s key reads differently from one refusing yours.

A report supports research triage; it is not technical, legal, regulatory, investment or freedom-to-operate due diligence. A valid citation ID does not establish that the source supports the claim.

Architecture

Topic / PDF + optional Decision Context
                   │
Deterministic retrieval → validation → frozen source registry
                   │
         ┌─────────┼─────────┐
     Academic    Patent    Market
         └─────────┼─────────┘
                 Writer
                   │
                 Reviewer
                   │
                  Scorer → deterministic weighted total
                   │
        Shared run artifacts + final run status
                   │
       FastAPI / browser / CLI / recovery

The three evidence specialists run in parallel; Writer, Reviewer and Scorer follow in sequence. A stage is not a promise of exactly one model request. See the code map and contribution rules before changing orchestration.

Measured results

The frozen baseline contains 10 topics × 3 live repetitions:

These are different checks, not a combined accuracy score. Expected TRL ranges were adjusted after early observations, so this is not independent held-out validation. The uncited-numeric proxy does not measure all hallucinations. Seven of ten topics met their TRL range in all three runs.

Held-out check. On ten new topics with ranges fixed from external evidence before any run (qwen3.7-plus, 2026-10-18), 21/30 runs fell in range, the pre-registered “supported” threshold. Scores were compressed toward 7, though: no run scored above 7, so mature, deployed technologies were under-scored. See the held-out result.

A second held-out cohort ran on 2026-10-19 with qwen3.8-max and the revised market retrieval. 25/30 runs fell in range, and 11 runs scored 8–9 on proven technologies. Low-maturity topics were not inflated, though the two lowest ones were scored below their floor. The topics and the model also changed, so the improvement is not attributed to any single change. Treat TRL as a coarse signal. See the v2 result.

Two later models were read against the same v2 ranges, one run a topic. deepseek-v4-pro on all ten topics (2026-11-08): 8/10 in range, both misses below their floor. deepseek-v4.1-flash-1017 on five of them (2026-11-11): 4/5 in range, the miss below its floor. These are spot checks, not calibration measures: one run a topic cannot tell the model from the day’s evidence. See the ten-topic result and the five-topic result.

The public deployment rotates through Model Studio’s free-quota models and is not pinned to any of these four; the one it currently requests is in /health as llm_model, and each run page names the model that produced it under “Technical details”. A model outside the four has no held-out reading. The form that asks for a model name shows the same four figures.

Other completed evidence:

• 90-cell topology ablation: the four-node arm used 54.89% fewer median tokens and 47.03% lower median cost than the six-node arm. Six nodes were not established as universally necessary.
• Five-reviewer utility study: 20 eligible judgments; the registered success rule failed despite a 6:4 full-workflow preference in each round.
• Two target-user pilot: both retained DEFER and answered MAYBE to reuse; neither checked external sources. This does not establish product adoption.
• Recovery: 30/30 offline fault-injection recoveries completed; one production recovery reused four committed nodes. This is not an exactly-once or general cost-saving guarantee.
• Runtime RTI02: one normal Qwen completion passed 12/12 primary final-status checks, with a disclosed minor observer-cadence deviation. Timeout/fallback paths and general report quality were not validated by that run.

Protocols, source artifacts and limitations are linked in the current evidence ledger.

Quick start

Use Python 3.11 or 3.12 for the CI-tested environment and uv. Dependency installation needs network access; the default test suite does not call providers.

git clone https://github.com/shuxiachai/academic-commercialization-agent.git
cd academic-commercialization-agent
uv sync

Copy .env.example to .env. For Qwen, set these values and replace the placeholders locally; do not commit or share keys:

LLM_PROVIDER=qwen
DASHSCOPE_API_KEY=your-key
QWEN_MODEL=qwen3.8-27b
QWEN_API_BASE=https://dashscope.aliyuncs.com/compatible-mode/v1
TAVILY_API_KEY=your-search-key

Choose the endpoint matching your operator account/region. This project’s browser BYOK default uses the China-region endpoint. With multiple LLM keys present, set LLM_PROVIDER explicitly; otherwise auto-selection is DeepSeek → Qwen → Anthropic → OpenAI. Tavily takes precedence over Serper when both search keys are present. Remove unused placeholder keys.

uv run uvicorn api.main:app --reload
# Browser: http://localhost:8000  (sample reports: http://localhost:8000/samples)
# Alternative CLI (starts real provider work):
uv run academic_agent --topic "solid-state batteries for electric vehicles"

Real analysis incurs provider usage. Duration is provider-bound: observed Qwen completions include 306 and 885 seconds, not a promised three-minute SLA. See the operating guide for HTTP endpoints, output files, Docker, access codes, BYOK, tracing and recovery.

Security and deployment

Three ways in, chosen per deployment: no gate (local use), operator access codes billed to the deployment, or email-and-password accounts that pay with the keys they add after signing in (how). An account’s keys stay in its browser and are never stored by the server. Each account may have one paid operation going at a time and ten a day by default, so one account cannot hold the host’s capacity (settings). Every deployment serves a privacy and terms page at /privacy that says what is stored and who else sees a topic.

The operator has a console at /admin, open only to admin accounts, for the service’s state, every account, password resets, disabling an account, and the account backups. The service backs up its account database daily and keeps the newest fourteen copies, and can tell a Feishu or WeCom group robot when a backup fails, the maintenance loop stops, or the deployment’s own key, quota or billing makes runs fail. With an email service set up (Resend), a new account proves its address through a one-time link before it can start paid work, and a forgotten password is reset by a link sent to the same address, and an account can ask to be emailed when an assessment of its ends. Without an email service the operator resets passwords.

Run URLs carry 128 bits of randomness and act as read capabilities: anyone with the full URL can read that run until retention removes it. Changing a code-owned run additionally requires its owner/admin code; ownerless BYOK runs have no second server-side identity. Do not publish private run URLs.

Before public deployment, configure access control, paid-operation limits, retention and persistent storage. PDF extraction is a paid operation too. Use one application replica / one Uvicorn worker: in-memory ownership and file-backed quotas are not a distributed queue. See deployment controls and checkpoint recovery.

Tests and benchmark

The pre-consolidation baseline at 0fdaa76 passed 2071 tests and 678 subtests. CI covers Linux/Windows × Python 3.11/3.12, latest Ruff, narrow Pylint, an 85% coverage floor, zero-provider Chromium smoke, and Docker. These do not constitute a paid-provider production SLO.

uv run pytest -q
uv run --with ruff ruff check .
# Scheduling preview only; no provider requests:
uv run python benchmark.py --dry-run

Benchmark

New measurements use immutable versioned batches; historical outputs are not overwritten. Reuse requires an explicit matching batch and intact artifacts. Reported costs cover Crew nodes, not helper/PDF/search charges. See the execution and accounting contract.

The benchmark guide explains repetitions, frozen evidence and paid-run precautions. Do not change scoring rules to improve these historical numbers.

Repository layout

The application is in a handful of directories. Most of the Python files at the repository root are something else: frozen experiment runners.

The experiment files are frozen where they are: their protocols lock paths and hashes, and the layout they recorded before 2026-10-31 is kept at the tag archive/root-experiments-2026-10 (see the experiment index). New tools go under scripts/.

Limitations and open questions

Open gaps: qualitative source entailment, independently measured decision utility, multilingual/short/non-technical benchmark coverage, legacy metadata read-state clarity, and single-replica scale. More agents, a vector database or Kubernetes would not by themselves resolve these. Accounts cannot yet reset their own password; a deployment without an email service also verifies no address, and should keep sign-up invite-only.

Market scoring. Market scorecards disclose unverified estimate comparability: the historical cap’s limitation is visible, but the underlying market-scoring policy is neither fixed nor validated. New results also show pre/post cap scores and the actual deduction; missing historical receipts cannot be reconstructed. The 20-source inspection still found no fully comparable real pair.

Tool Calling. Three capabilities have separate release boundaries:

• Saved-source location has an opt-in production page/API: at most one bounded Qwen selection from saved IDs/titles, then the exact saved text. It writes no research answer and runs no new search; owner authorization, transfer consent, paid admission and receipts apply (operating contract). One prepared production acceptance on 2026-09-26 delivered the chosen source and recovered its receipt after a redeploy (1,199 tokens, a USD 0.000764436 frozen-rate estimate); see the qualified acceptance record. That is one delivery observation, not unseen accuracy. Paid execution was closed again with zero feature budgets; it is not permanently enabled.
• Supplementary retrieval remains zero-call shadow mode in production; its latest unseen evaluation failed the admission gates.
• Generated evidence follow-up answers are experimental, not a production feature. Tool execution or valid JSON does not prove claim support.

Hypotheses, failures and closed batches are kept in the evidence ledger and experiment archive; they are not combined into a general accuracy score.

Documentation

• 中文项目说明
• Documentation map — reading paths, repository layout and the boundary between current guides, historical experiments and local/private material.
• Evidence ledger — measurements and what they cannot prove.
• Operating guide — setup, API, deployment and scoring.
• Contributing and AGENTS.md — tested constraints and rejected approaches.
• Experiment archive — unchanged protocols, results and errata; older numbers are historical snapshots.

Screenshots

Author and more projects

Created by shuxiachai. Contributors: Song Guoyu (@7776403yu), @xiaobaozhou165-sudo, Abhishek Vyas (@Baddevil512). Explore more projects on the author’s GitHub profile.

License

MIT. Contributions are accepted under the same license; see CONTRIBUTING.md.

For Chinese documentation, open README.zh-CN.md.