strix-halo-guide

Strix Halo Guide: AMD Ryzen AI MAX+ 395 Local LLM Setup & Benchmarks

By a contributor to llama.cpp, Lemonade, OpenAI’s .NET SDK and NVIDIA AICR

Runs models up to 284B on one 128GB mini PC, measured on a Beelink GTR9 Pro, with results from 13 systems and sources. Raw logs and caveats.

Independent guide, not endorsed by AMD, any OEM or the projects named above.

Quick Start (6 Steps)

Run a local LLM on the GPU of a 128GB AMD Strix Halo / Ryzen AI MAX+ 395 (Radeon 8060S) with Ubuntu 24.04, Ollama and Vulkan/RADV.

1. BIOS: set the UMA frame buffer to 512MB if offered, otherwise the lowest value your vendor allows (often 2GB). Leave IOMMU enabled/default.
2. Ubuntu: install Ubuntu 24.04 LTS directly on the hardware.
3. Install: run the setup script below. It adds the 128GB boot parameters amdgpu.gttsize=131072 ttm.pages_limit=31457280 amdgpu.cwsr_enable=0, upgrades Mesa/RADV and configures Ollama for Vulkan with OLLAMA_VULKAN=1, OLLAMA_IGPU_ENABLE=1 and HIP_VISIBLE_DEVICES=-1.
4. Reboot when the script prints REBOOT REQUIRED.
5. Check: the boot check and the known-answer check must both end in PASS.
6. Chat: run ollama run qwen3.6:35b-a3b, then confirm that ollama ps shows 100% GPU.

Another memory size (64/96/192GB)? The 128GB values do not carry over: use the manual route and its memory section. On Windows: the Windows route.

Setup Script

Step 3: install. Run this as your normal user, not as root. It stops at the first command that fails:

sudo apt update &&
sudo apt install -y git curl python3 zstd software-properties-common &&
cd "$HOME" &&
git clone https://github.com/hogeheer499-commits/strix-halo-guide &&
cd "$HOME/strix-halo-guide" &&
bash setup.sh

Steps 4 to 6: reboot, check, chat. Reboot if the script printed REBOOT REQUIRED, then run:

cd "$HOME/strix-halo-guide" &&
python3 scripts/check_local_ai.py boot &&
python3 scripts/check_local_ai.py answer &&
OLLAMA_HOST=http://127.0.0.1:11434 ollama run qwen3.6:35b-a3b

Good: both checks end in PASS and the model answers your question. Type /bye to leave the chat, then check the GPU:

OLLAMA_HOST=http://127.0.0.1:11434 ollama ps

It should show 100% GPU. An empty list means the model has already unloaded: run the chat once more and check again right away. CPU or a CPU/GPU mix means this check has not passed: see CPU fallback.

Keep Ollama local. It has no password. The script binds it to 127.0.0.1 and the answer check confirms that.

Working? Choose a model for your RAM and task or add browser chat. Stuck? Stop at the first error and use If a step fails; Get local AI running, step by step shows what every step should print.

Existing installations, version pinning, the curl | bash route and how to undo the changes: Setup Script Details.

What This Gives You

• A clear setup route. The settings that matter on this chip, from the BIOS memory option to the one Ollama setting without which measured builds quietly run on the CPU, in one step-by-step start route that shows the expected result of every step.
• A model for your RAM and task. What can I run with my RAM? gives a starting point from 32GB to 192GB and shows which route was tested here and which are candidates. Best Known Profiles has the measured settings and speeds.
• Know what’s tested. See which settings have been tested, what is still experimental, and which known failures you can avoid. If something goes wrong, use the troubleshooting steps. Every headline number links to its raw evidence or carries an explicit caveat.

Start Here

I want to…	Start here
Get local AI running	Install, get an answer and check the GPU
Choose a model or make it faster	Choose by RAM and task, then Best Known Profiles
Fix something that is not working	Troubleshooting
Decide what to buy	Choose a system for your task and Buyer Use Cases

Also: website · Qwen3.8 · reproduce a result · vendor/reviewer brief.

Use This Guide If

• You own, ordered, review, sell, or are evaluating a Strix Halo / Ryzen AI MAX system for local AI.
• You want a working route and the evidence needed to understand where it transfers—or fails to transfer—to another system.
• You need independent buyer guidance that can coexist with future disclosed vendor or affiliate relationships.

A complete, practical guide to running large language models locally on AMD Strix Halo / Ryzen AI MAX+ 395 systems with Radeon 8060S (gfx1151) and 96GB/128GB unified memory.

Covers BIOS config, Ubuntu 24.04/kernel setup, Ollama, llama.cpp Vulkan/RADV, ROCm/HIP experiments, vLLM notes, 70B/120B and selected 284B GGUF capacity evidence, benchmarks, raw logs, and reproducibility checks.

AMD now publicly frames Ryzen AI Halo-class systems as a local-AI and developer-platform direction. This repository is the independent practical layer: copyable setup, measured rows, raw evidence, failures, and community reproductions. It is not official AMD or OEM endorsement. See RYZEN_AI_HALO_CONTEXT.md.

Project website: https://strixhaloguide.com/. This GitHub repository remains the source of truth for setup commands, benchmark claims, and raw evidence.

What you get:

• A copyable Ubuntu + Vulkan/RADV setup for Ollama and llama.cpp.
• Workload-to-model/backend recommendations instead of one undifferentiated leaderboard.
• Direct, API/server, MTP/speculative, concurrency, capacity, power, NPU, RPC, first-party, and community claims kept separate.
• Headline rows linked to CSVs, raw logs, charts or explicit n/a, plus preserved failures and corrections.
• Cross-OEM evidence covering 13 systems or independent sources from 10 credited community benchmark contributors.

Measured primarily on one Beelink GTR9 Pro. Community results are kept separate from local headline claims. This repository ships docs, scripts, data, and charts only; no .exe, binary .zip, browser extensions, or model weights. Raw evidence, commands, caveats, and corrections are linked so results can be checked instead of taken on trust.

Quick Start | Qwen3.8 | Profiles | Current Models | Evidence | Community | Reproduce | Contribute

If this guide saves you setup or research time, a GitHub star helps other Strix Halo owners find it; a reproducible result or correction helps even more.

Need	Source of truth
Full beginner route	Install, check the result and fix errors; manual reference
Current Qwen3.8 decision	QWEN38_STRIX_HALO.md and data/qwen38_route_matrix.csv
Known-good workload profiles	BEST_KNOWN_PROFILES.md and data/best_known_profiles.csv
Auditable benchmark claims	Headline Evidence, data/headline_claims.csv, and data/raw/
Cross-OEM buyer evidence	SYSTEM_EVIDENCE_MATRIX.md, COMMUNITY_RESULTS.md, and BUYER_PATH_VALIDATION.md
Current blockers and test priorities	CURRENT_MODELS.md, ROCM_VLLM_BUGWATCH.md, and data/current_test_queue.csv

Maintenance And Evidence Status

Maintenance update: November 8, 2026. Model choices by RAM and task, corrected setup evidence, selected release and issue states, 192GB order phases and service details: MAINTENANCE_REVIEW_2026-11-08.md. The November 9 broad evidence review follows this scoped update; historical measurements keep their original dates.

Current evidence state, November 9, 2026 review: Qwen3.8 27B is measured through the official Ollama route, labelled as Ollama-default MTP drafting; a matched control on Ollama 0.32.15 (2026-09-26) measured 12.89 t/s without drafting versus 22.71 t/s with default MTP. See the active-evidence review; the September 26 review keeps its original wording. The August 30 b10687 Vulkan sentinel and Flash-Next scout are indexed with their own stack and caveats; see the measured update. The November 9 availability check records Ollama 0.40.15, llama.cpp v0.9.0 and numbered prerelease b12211 as available, not measured or qualified here. The security recheck adds published reports affecting older pins, including Ollama’s pull API and llama.cpp parser/GGUF/RPC paths. Storefront blocks and advisory-feed limits are explicit in the review. The machine-readable freshness record is data/public_state.json.

Older changes are in the Changelog.

20-Second Summary

Question	Current answer
Best beginner route	Ollama with Vulkan/RADV. The fully reboot-qualified general baseline remains 0.31.2; Qwen3.8 is separately measured on 0.32.13. Available Ollama 0.40.15 (November 9 check, not qualified; 0.34.2 passed isolated controls on September 19) still needs the controlled upgrade/reboot matrix; the revised installer itself is not yet fresh-install/upgrade qualified.
Current Qwen3.8 route	Official Qwen3.8 27B Q4_K_M measured 292.49 prompt t/s and 20.42 generation t/s through the Ollama API with Ollama-default MTP drafting (draft_num_predict 4; not a no-draft result); a separate matched control on Ollama 0.32.15 (2026-09-26) measured 12.89 t/s without drafting versus 22.71 t/s with default MTP; image, tools, thinking, and exact retrieval through 50,059 prompt tokens passed. Read the route comparison.
Fast direct 30B-class route	Qwen3-Coder 30B-A3B Q4_K_S reached 100.99 tg128 on official b9851; it is a speed-first quant, not the balanced default.
Largest direct GGUF tested	DeepSeek V4 Flash 284B UD-IQ2_XXS loaded as a 90.86GB low-bit artifact and measured 13.27 tg128; capacity proof, not broad quality.
Experimental server frontier	Repeat-confirmed local MTP/server profiles reach 101-141 t/s, but prompt shape, draft acceptance, runtime, and quant are part of each claim.
Cross-system confidence	13 systems or independent sources and 10 credited community benchmark contributors, with community data separated from first-party Beelink rows.
Trust model	Every headline maps to structured/raw evidence or an explicit caveat; failed paths and negative results remain public.
Commercial independence	No affiliate links are present as of September 19, 2026. Future affiliate/vendor support must be disclosed and cannot buy ranking, positive conclusions, or removal of accurate negative findings.

What You Can Run: Quick Snapshot

This is the quick “what can I actually run on my AI PC?” view. It is not the full benchmark list; see What You Can Run for more models and Headline Evidence for the audit trail.

What you want to do	Measured local result	Practical takeaway	Evidence
Fastest direct 30B-class Qwen MoE row	Qwen3-30B-A3B-Instruct-2507 IQ4_XS: 100.04 t/s direct llama.cpp Vulkan/RADV on b9467; b9544 control measured 103.18 tg128 r10	First local direct llama-bench row above 100 t/s. Treat it as a separate general-instruct Qwen route, not as a Qwen3-Coder replacement or balanced-default claim.	headline claims, raw r50, b9544 control
Fastest current small-MoE scout	LFM2.5 8B-A1B Q4_K_M: 168.96 tg128 in pp512/tg128, 170.02 t/s generation-only, b9544 control at 176.48 tg128 r10	Shows how fast newer small active-parameter MoE routes can be on Strix Halo. Do not compare it as a 30B-class coding/reasoning replacement.	headline claims, raw latest/int-dot, b9544 control
Fastest local coding speed	Qwen3-Coder 30B-A3B Q4_K_S: 100.99 t/s direct llama.cpp Vulkan/RADV on the official b9851 release binary; older strict-clean b9179 row measured 98.51 t/s	Speed-first quant candidate. Use it when raw t/s matters and you accept the quality tradeoff. This is direct llama-bench, not MTP/server speculation.	headline claims, b9851 raw r50, older strict-clean raw r50
Fast balanced local coding model	Qwen3-Coder 30B-A3B UD-Q4_K_XL: 96.76 t/s direct llama.cpp Vulkan/RADV on current b9049	Coding-speed row for the balanced quant: a speed measurement, not a coding-quality ranking. Third-party quality signals and public scores are in docs/models.md.	headline claims, raw run
Newer Qwen coding model	Qwen3-Coder-Next 80B-A3B IQ4_XS: 61.91 t/s direct llama.cpp Vulkan/RADV on b9467	Modern coding-model row for people who want current Qwen Coder-Next rather than the older 30B speed headline. Use it for capability/currentness, not maximum raw t/s.	benchmarks CSV, raw run
Easy local chat setup	Qwen3.6 35B-A3B Q4_K_M: 60.57 t/s warm API generation through the normal Ollama 0.31.2 system service with OLLAMA_IGPU_ENABLE=1; vision, service restart, and full-host reboot persistence passed	This is the copyable default for model pulling, Open WebUI, vision, and simple local chat. A later controlled local-binary comparison measured 0.31.1/0.31.2/0.32.0 at 72.55/73.19/73.20 t/s, so do not treat the earlier 60.57-versus-71.82 gap as a version-wide regression.	headline claims, raw service run, raw controlled comparison
Fine-tune, export, and reload a local model	Pinned ROCm 7.2 Unsloth route: Radeon GPU gate, one SFT step, checkpoint inference, Q4_K_M GGUF export, ROCm llama.cpp inference, and post-restart artifact load all passed	End-to-end developer workflow evidence on a retail box. The Qwen3 0.6B one-step run is a plumbing smoke, not a quality or performance headline.	Unsloth guide, raw evidence
Fast all-rounder direct path	Qwen3.6 35B-A3B UD-Q4_K_M: 62.56 t/s direct llama.cpp Vulkan/RADV on current b9049	Use this when you care more about speed and control than the easiest UI.	headline claims, raw run
Fastest Qwen3.6 direct path	Qwen3.6 35B-A3B Q4_0: 81.30 t/s direct llama.cpp Vulkan/RADV on current b9049	Speed-first option. Use the default/balanced quant if quality matters more than raw t/s.	max campaign, raw run
Qwen3.6 27B dense control	Official Qwen3.6 27B MTP Q8_0: 7.61-7.74 t/s without MTP, 14.59-14.69 t/s with the best MTP setting; a direct b9467 llama-bench follow-up measured 7.70 t/s tg128	Useful practical row for people comparing 27B versus 35B-A3B. It runs, but this dense Q8 route is much slower than the 35B-A3B MoE paths and is not a speed candidate.	Performance notes, MTP CSV, raw b9235, raw latest, raw b9467
Experimental Qwen3.6 MTP server path	Qwen3.6 35B-A3B MTP IQ4_XS-Q8nextn: 101.16 t/s best local Beelink six-prompt average on b9360, with t16 repeats at 101.15 / 101.10 / 101.06 t/s; first GMKtec community reproduction reached 93.29 t/s on b9235	Advanced speculative-decoding route. Useful if you are testing a local API server; keep separate from direct llama-bench.	MTP notes, MTP CSV, local raw, GMKtec raw
Experimental Gemma 4 QAT MTP server path	Gemma 4 26B-A4B QAT UD-Q4_K_XL + matched Q4_0 MTP head: 110.00 t/s best repeat, 107.42 t/s T3-only repeat, 102.69 t/s cold repeat, 73.96 t/s no-spec baseline on ac4cddeb0	Historical server throughput, not a practical recommendation: six sampled saved coding answers failed their tasks in both no-spec and MTP records; see the saved-output review. These are server/speculative results, separate from direct llama-bench.	headline claims, MTP CSV, warm raw, cold raw, T3-only raw
Official Gemma 4 31B QAT multimodal route	Official Q4_0 GGUF: 308.28 pp512 / 11.38 tg128 direct on b10066; narrow text, STRIX 395 vision, and native calculator tool-call smokes passed	Current dense Google-model compatibility route for buyers who value text, vision, and tools over maximum decode speed. The matched Q8_0 DFlash sidecar loaded but was slower on the measured 5.5K/21.9K synthetic prompt shapes because acceptance stayed low.	benchmarks CSV, MTP CSV, raw evidence
Experimental CHADROCK ROCmFP4 MTP server path	CHADROCK ACE/SABER 35B ROCmFP4 through ciru-ai/ROCmFPX: 141.37 t/s mean across three repeats on the exact 3946-token reference profile, with 100% mean draft acceptance	Fastest repeat-confirmed server/speculative reference profile in the guide, but not a universal speed. Separate 1K/8K/16K profiles measured 78.00/83.85/107.23 t/s as acceptance changed. Treat as advanced ROCmFPX/CHADROCK evidence, not a direct llama-bench replacement.	ROCMFP4_CHADROCK.md, raw stability profile, MTP CSV
Frontier-size agent server	Step 3.7 Flash 198B-A11B ROCmFPX Q3 plus Q8 MTP draft: 23.84 t/s matched 4K no-spec baseline, 34.50 t/s 4K MTP, 33.83 t/s 16K MTP, native tool-call pass, and 256K allocation	Shows the capacity and agent value of 128GB unified memory rather than chasing the fastest small-model row. Advanced pinned runtime; server/MTP result, and the 48K row has one repeat.	ROCmFPX guide, raw evidence, MTP CSV
80B MoE coding/reasoning experiments	Qwen3-Next 80B-A3B UD-Q4_K_XL: 59.06 t/s direct llama.cpp Vulkan/RADV on b9172	Historical May 80B Qwen-family path measured here; use when model size and 256K context matter more than smallest footprint.	headline claims, raw r20
Open-weight 120B reasoning model	gpt-oss-120b MXFP4: 55.57 t/s direct llama.cpp Vulkan/RADV on current b9049	128GB unified memory can run a 117B-parameter MoE locally; this is speed evidence, not a model-quality eval.	headline claims, raw run
Current 120B-class GGUF capacity route	Nemotron 3 Super 120B-A12B UD-IQ4_XS: 18.43 t/s direct llama.cpp Vulkan/RADV, b9544 control at 18.93 tg128 r3	Answers a different buyer question: yes, a current 120B-class MoE GGUF route can run directly on one 128GB Strix Halo box.	headline claims, raw latest/int-dot, b9544 control
Current NVIDIA Omni/FP4 route	The same Nemotron 3 Nano Omni MXFP4 artifact improved from 56.56 tg128 on b9747 to 64.26 on b10034; a separate NVFP4 + F16 projector route measured 53.21 tg128 and correctly read STRIX 395 from an image	Shows both current runtime maintenance and a first-party image-capable Nemotron route. The OCR check is not broad vision/audio/video validation and neither row replaces the Qwen speed headlines.	benchmarks CSV, raw MXFP4 sentinel, raw multimodal scout
Current agent/reasoning scouts	Nemotron Cascade 2 30B-A3B IQ4_XS: 78.95 tg128; Qwen AgentWorld 35B-A3B UD-IQ4_XS: rate withheld pending retained-run provenance with a correct terminal-world smoke and 128K allocation pass	These answer current-model and agent-use-case questions without pretending every new model is a speed headline. Cascade’s forced no-think prefix did not hide reasoning; AgentWorld’s 128K result is an allocation smoke, not a filled-context quality claim.	CURRENT_MODELS.md, Cascade evidence, AgentWorld evidence Curator: exact retained-run disagreement; rate provenance pending.
Local API for tools or several clients	Qwen3-Coder 30B-A3B b9979: 228.18 aggregate t/s stock at np8; opt-in density+dense16 reached 234.12 at np9, while density alone reached 266.07 at np16	A software dispatch cliff, not memory capacity, can limit multi-user value. Keep stock for low concurrency; advanced users should compare density Vulkan and ROCm at their exact target.	MOE_CONCURRENCY.md, summary CSV, 30B chart
FP16 vLLM at 8-16 concurrent requests	Official ROCm 7.14 image, PyTorch 2.11, Qwen3-0.6B: TORCH_BLAS_PREFER_HIPBLASLT=1 improved aggregate throughput by 40.50% / 38.96% / 41.54% at concurrency 8/9/16	This reproduces AMD’s Ryzen AI batch-8+ workaround without changing the host. It is a small-model FP16 server A/B, not a direct GGUF or 27B/35B claim; concurrency 4 was slightly slower.	ROCm/vLLM notes, processed A/B, raw evidence
Historical ROCm/HIP llama.cpp allocation smoke	Official b10046, Qwen3-0.6B Q8_0: 4666.05 pp512 / 208.73 tg128; full 120,124 MiB free UMA detected and ROCm_Host buffers allocated	Reproduced full-UMA discovery without HSA_OVERRIDE_GFX_VERSION, but open issue #26209 and the model-scoped v0.4.1 mitigation controls do not make the tiny smoke qualify long-context, multimodal, multi-slot, or practical-model correctness.	ROCm/HIP notes, raw evidence
Long documents or codebase context	Qwen3.6 35B-A3B: 32.23 t/s decode after a filled 128K KV cache	Long-context use is possible, but prompt ingestion cost matters.	filled KV CSV, chart
Large-model proof point	MiniMax M2.7 230B-class MoE loaded and generated locally; Llama 4 Scout 109B measured 18.32 t/s historically	128GB unified memory makes very large local models practical on one compact PC, but capacity and speed are different wins.	CURRENT_MODELS.md, benchmarks CSV

Use This If You Want

Goal	Start with	Why	Evidence
Easiest local chat	Ollama 0.31.2 system service with Vulkan/RADV	normal install path; model pulling, vision, restart/reboot persistence, and Open WebUI compatibility. Local is not automatically private: prompts can end up in logs and caches, see SECURE_LOCAL_AI.md	60.57 t/s fully qualified service run; controlled isolated 0.31.1/0.31.2/0.32.0 binaries later measured 72.55-73.20 t/s, data/benchmarks.csv
Fast coding or scripts on one machine	llama-server Vulkan/RADV	fastest measured Qwen3.6 path at 1-4 parallel requests	SERVER_SHOOTOUT.md
Speculative decoding experiments	llama-server MTP on current master / ROCmFPX	measured server speedups on Qwen3.6 MTP GGUFs, Gemma 4 QAT matched MTP heads, and CHADROCK ROCmFP4; Qwen3.6 reached about 101.1 t/s on b9360, Gemma 4 26B-A4B QAT reached 110.0 t/s best-repeat, and the exact CHADROCK reference profile averaged 141.37 t/s over three repeats at 100% draft acceptance	MTP_SPECULATIVE_DECODING.md, ROCMFP4_CHADROCK.md, data/mtp_speculative.csv
Several local tools or users hitting one API	Start with stock Vulkan up to 8; above that, compare opt-in density Vulkan and Lemonade ROCm on the exact model	b9979 repeats show density recovers the Vulkan np9 cliff, but the backend winner differs between the tested 30B and 80B models	MOE_CONCURRENCY.md, data/moe_density_gate_summary.csv
Long local documents or codebase context	llama-server Vulkan/RADV first, test ROCm/HIP for prompt-heavy ingestion	128K prompt plus generation completed; HIP can win prompt processing	data/filled_kv_decode.csv, BACKEND_CROSSOVER.md
Prompt-heavy ROCm experiments	Keep a reproducible ROCm/HIP + ZenDNN path available, but do not treat it as a decode-speed headline.	A community Beelink GTR9 Pro on CachyOS / kernel 7.0.11 / ROCm 7.2.4 / ZenDNN measured Qwen3.6 27B MTP UD-Q6_K_XL at 303.20 pp5000 on ROCm versus 155.89 pp5000 on Vulkan, while decode stayed around 8 t/s on both backends.	BACKEND_CROSSOVER.md, COMMUNITY_RESULTS.md
NPU sidecar work	Treat the NPU as a possible low-overhead sidecar, not as a replacement for the main iGPU LLM path.	A ciru-ai GMKtec EVO-X2 NixOS artifact kept IOMMU enabled and measured only +3.29% main 64k iGPU workload latency with concurrent NPU load, versus +68.96% with a comparable iGPU auxiliary load. The same package reports FastFlowLM-NPU LFM2.5 1.2B at 32k around 1646 prompt tok/s, 38.18 decode tok/s, and about 2.09GiB RSS.	COMMUNITY_RESULTS.md#gmktec-evo-x2-nixos--npu--rocmfp4-evidence-package, data/community_ciru_evox2_metrics.csv
vLLM-style serving experiments	Start with isolated ROCm containers; for ROCm 7.14 plus PyTorch before 2.14, A/B TORCH_BLAS_PREFER_HIPBLASLT=1 at batch 8+	The local Qwen3-0.6B FP16 A/B measured about 39-42% more aggregate throughput at concurrency 8/9/16. This validates the setting and container path, but no practical 27B/35B vLLM throughput claim exists yet.	processed A/B, ROCM_VLLM_BUGWATCH.md
Sustained inference on a Corsair/Sixunited AXB35 system	Verify the expected EC/fan module and dependent services after every kernel update before applying a clock cap.	A strict three-system campaign found a useful 2400 MHz fleet tradeoff, but historical journals also showed missing ec_su_axb35 modules and failed fan services after updates. The root cause remains unresolved and the cap is not universal.	THERMAL_STABILITY.md, data/community_thermal_sclk.csv

Community-Tested Rules Of Thumb

These are the practical decisions extracted from the primary Beelink runs plus community reports from Corsair, GMKtec, MS-S1-Max, Nimo, a second Beelink owner stack, and a ciru-ai GMKtec EVO-X2 NixOS/NPU artifact. Use them to avoid retesting dead ends first; follow the evidence links if your setup differs.

Situation	Do this first	Why	Evidence
One Strix Halo AI PC	Use Vulkan/RADV for GGUF chat, coding, and generation-heavy inference.	It is the fastest measured practical path for the main Qwen MoE rows.	headline claims, COMMUNITY_RESULTS.md
Native Linux on another Strix Halo vendor	Expect the same performance class if backend, model, quant, and command match.	GMKtec EVO-X2 96GB on Ubuntu 26.04, Mesa RADV 26.0.3, and llama.cpp b9156 reproduced the guide’s Qwen3.6 UD-Q4_K_M row within -0.8% pp512 and -1.7% tg128.	COMMUNITY_RESULTS.md, #16
Comparing Qwen3-Coder rows	Preserve the exact model hash, build and command flags before calling one system faster.	The August 26 GMKtec follow-up supplies hashes and direct rows of 99.08 tg128 for Q4_K_S and 96.75 for UD-Q4_K_XL. It strengthens portability evidence; runtime, model and batch differences still prevent an OEM ranking.	Community follow-up, raw bundle
Handling “older model” criticism	Add current-model rows, but keep speed and capability separate.	Qwen3-Coder-Next IQ4_XS runs locally at 61.91 t/s on the Beelink with b9467. That is a useful modern coding-model result, but it does not replace the older Qwen3-Coder 30B Q4_K_S speed-first headline.	raw Qwen3-Coder-Next run, data/benchmarks.csv
Tuning llama.cpp batch flags	Check that -ub is not larger than -b before any other tuning.	-ub above -b is silently clamped to min(n_batch, n_ubatch) with no warning. A community GTR9 Pro report measured +28.9% pp512 from correcting -b 256 -ub 1024 to -b 2048 -ub 512, and a further +16.9% pp512 / +3.1% tg128 from upgrading stock Mesa 25.2.8 to kisak 26.1.7 (no reboot needed; RADV is userspace).	COMMUNITY_RESULTS.md, raw note
Seeing a direct 100 t/s 30B-class Qwen result	Check the exact model, quant, build, and claim category before comparing.	There are now two separate first-party direct 100-class Qwen rows: Qwen3-Coder 30B-A3B Q4_K_S at 100.99 t/s on official b9851, and Qwen3-30B-A3B-Instruct-2507 IQ4_XS at 100.04 t/s on b9467 with a b9544 control at 103.18 t/s. The Qwen3-Coder row is speed-first, not the balanced default; the 2507 row is a separate general-instruct model.	b9851 Qwen3-Coder raw, 2507 raw scout, headline claims
Seeing Qwen3-Coder around 100 t/s on another Strix Halo box	Treat it as a tuned-system clue, not a default claim. Capture thermals, power policy, Vulkan device line, glslc --version, driver/toolchain details, and exact command.	A Reddit GMKtec EVO-X2 report saw most Q4_K_S -p 0 -n 128 runs around 99.90 t/s and a best 100.0 t/s after repasting, reseating memory pads, and using GPU high plus CPU EPP performance. Local Beelink b9467 follow-ups stayed around 95.27-96.72 t/s, so thermals/power/toolchain still need to be separated before calling this generally reproducible.	COMMUNITY_RESULTS.md#reddit-gmktec-evo-x2-tuned-100-ts-report, PERFORMANCE_NOTES.md#vulkan-integer-dot-and-100-ts-reproduction-status, raw reproduction
Starting on Windows	LM Studio Vulkan is now a documented Windows path, but keep it separate from Linux llama-bench.	The first Windows MS-S1-Max report measured a 89.49 tok/s script average through LM Studio with n_parallel=4 and 262K context; the long 512-token prompt rows were around 69-70 tok/s. This is useful Windows buyer evidence, not a same-machine Windows-vs-Linux comparison.	COMMUNITY_RESULTS.md#windows-lm-studio-ms-s1-max-report, raw Windows report
Evaluating a compact non-Beelink chassis	Look for setup metadata, thermal context, and large-model feasibility, not only headline t/s.	The Nimo AI Mini PC issue #4 bundle adds Ubuntu 25.04 / Mesa 25.2.8 / ROCm rows, Qwen 122B-class serving, StepFun 198B-class serving, Qwen3-Coder-Next server rows, DFlash negative/control evidence, Gemma 4 QAT/MTP assistant-head follow-up data, and supplemental fan/power/temperature telemetry.	COMMUNITY_NIMO.md, data/community_nimo_issue4.csv, raw Nimo bundle, Gemma QAT follow-up
Choosing an IOMMU policy	Keep IOMMU enabled/default for normal systems, NPU work, and mobile suspend. Use amd_iommu=off only for the optional always-on desktop benchmark profile.	The Beelink headline profile preserves the measured performance route, while the safer general default avoids silently losing the NPU or deep laptop/tablet sleep. ciru-ai’s GMKtec EVO-X2 NixOS artifact also shows an IOMMU-on stack where the NPU can run sidecar work with low measured main-workload impact.	setup policy, COMMUNITY_RESULTS.md#gmktec-evo-x2-nixos--npu--rocmfp4-evidence-package
Testing MTP/speculative decoding	Treat MTP as an advanced server route, not a direct benchmark replacement.	The Qwen3.6 MTP IQ4_XS-Q8nextn route has a local b9360 rerun at 101.16 t/s and a GMKtec b9235 reproduction at 93.29 t/s. The Gemma 4 26B-A4B QAT route adds a current Google-model matched-head example at 102.69-110.00 t/s. The exact CHADROCK ACE/SABER reference profile averaged 141.37 t/s over three repeats at 100% acceptance, while lower-acceptance shapes were much slower.	MTP_SPECULATIVE_DECODING.md, ROCMFP4_CHADROCK.md, data/mtp_speculative.csv, #18
The model fits on one Strix Halo box	Do not use llama.cpp RPC for raw single-stream speed.	2-node RPC lost about 14-22% tg128 on fits-on-one models; 3-node was slower again.	COMMUNITY_RPC.md, data/community_rpc.csv
A huge GGUF does not fit on one box	Try ROCm RPC first, starting with the smallest node count that fits.	In the tested MiniMax-M2.7 140.8GB case, one box failed, 2-node ROCm worked, and 3-node ROCm was slower. This is a capacity rule from that case, not a universal speedup rule.	COMMUNITY_RPC.md
Testing huge MoE capacity on one box	Separate “loads and processes prompts” from “fast generation”.	A Corsair AI Workstation 300 ai-2 community row loaded MiMo-V2.5 UD-IQ2_M, a 310B-total / 15B-active GGUF route, and measured 30.65 pp512 with wall-power/GPU telemetry. The pasted CSV row has n_gen=0, so it is capacity/telemetry evidence, not a tg128 speed claim.	COMMUNITY_RESULTS.md#corsair-ai-workstation-300-mimo-v25-capacity-row, #26
Building a USB4 Strix Halo cluster	Start with MTU 9000 and pm_qos_resume_latency_us=100.	MTU 9000 beat 1500 and 65520; pm_qos added about +2% tg128 with only about 1.5 W idle cost per toggled box in the community report.	USB4_CLUSTER_TUNING.md
Choosing Qwen3.6 quantization	Use Q4_K_M/UD-Q4_K_M for balanced defaults; use Q4_0 only when speed matters more than quality.	Q4_0 was faster locally and in the Corsair report, but this guide has not made a model-quality claim for it.	COMMUNITY_RESULTS.md, data/community_results.csv
Serving an interactive local API	Prefer single-box llama-server when the model fits.	Community llama-server TTFT was about 201 ms on 1 node versus about 301 ms on 2-node RPC, with higher RPC variance.	COMMUNITY_RPC.md, data/community_rpc_server.csv
Comparing your t/s to this guide	Treat about 2% tg128 spread as normal between well-matched Strix Halo systems.	Three matched Corsair boxes showed 0.11% pp512 spread and 2.05% tg128 spread; the GMKtec native Qwen3.6 row landed within 2% of the Beelink row.	COMMUNITY_RESULTS.md
Estimating electricity and heat	Use wall-power rows as workload-specific context, not universal TDP.	Community wall-power data measured about 150 W / 1.6 J/token for Qwen3-Coder, 148 W / 2.0 J/token for Qwen3.6, 174 W / 3.1 J/token for gpt-oss-120b, and 137 W / 3.4 J/token for Qwen3-Coder-Next.	COMMUNITY_RESULTS.md#whole-system-power, data/community_power.csv
Seeing a Vulkan/RADV failure on a huge MoE	Check for per-buffer allocation limits, not only total memory.	MiniMax-M2.7 hit the same 830472192-byte RADV allocation failure on 1-node and RPC follower paths.	data/community_rpc_failures.csv, COMMUNITY_RPC.md

Results Wanted

Have a Strix Halo / Ryzen AI MAX system? Please share results, even if they are slower, failed, or contradict this guide.

• Open a benchmark report with your system, BIOS UMA setting, kernel, Mesa/ROCm versions, backend, model, command, CSV/raw output, and notes.
• Open a power / efficiency report if you can measure wall power, UPS power, or validated board power during the same benchmark command.
• Open a model request if there is a model/backend combination that should be tested.
• Use Discussions for setup questions, comparisons, and early results that are not ready for a benchmark issue yet.
• Power telemetry is useful if you have reliable wall-power data; it helps turn raw tokens/sec into tokens-per-watt context for buying, cooling, and always-on server decisions.
• Current community reports live in COMMUNITY_RESULTS.md, COMMUNITY_NIMO.md, COMMUNITY_RPC.md, USB4_CLUSTER_TUNING.md, and the community_* CSVs listed in data/README.md.
• See SHARE.md for short Reddit/HN/forum text and the current social preview image if you want to share the guide.

For Vendors, Partners, And Reviewers

This guide is primarily a technical resource for AMD Strix Halo local-AI users. For vendors, reviewers, and partners, it also documents a reproducible way to reduce buyer setup friction and validate real-world local-AI use cases without weakening independent benchmark discipline.

Start with the public vendor and reviewer overview, then inspect the linked GitHub evidence, raw artifacts, negative results, disclosure policy, and upstream contribution record here.

Start with ONE_PAGE_BRIEF.md and PARTNERSHIP.md. Supporting docs cover SPONSORSHIP.md, VENDOR_DISCLOSURE.md, BUYER_USE_CASES.md, SPONSOR_ROADMAP.md, and TRACTION.md. How vendors can report a factual error, how vendor responses are recorded and how hardware provenance is disclosed: How We Work With Vendors.

No affiliate links are present as of September 19, 2026. The links added on October 1, 2026 were checked by URL pattern for tracking parameters and affiliate-network domains and none was found (the destinations themselves were not inspected). If monetized product links are added later, each relevant link will be labeled and recorded in the public affiliate link registry. Affiliate commission does not determine benchmark conclusions, product inclusion, or ranking.

Best Current Setup Tested Here

These are the AMD Strix Halo / Ryzen AI MAX+ 395 configurations behind this guide’s measured results. Pick the starting point that matches what you want to do; each result applies to the exact versions it was recorded with.

New to local AI? Follow the complete start route. It keeps installation, the first answer, the GPU check and help with errors on one page, and treats a browser interface as the next step for daily use. The measured profiles below were recorded on an existing setup; they are not proof that a fresh run of the revised installer works.

Choose the backend by what you are trying to do:

Use case	Do this first	Why
You want to get local AI running	Follow the complete start route.	Installation, a visible answer, a GPU check and help with errors on one page.
You want to reproduce the headline speed rows	Use Reproduce One Headline Result.	Exact model, quant, build, and command matter for benchmark comparisons.
You want a local API server or MTP tests	Read MTP/speculative decoding and use llama-server.	Supports serving, batching, long-context tests, and speculative decoding.
You have many parallel local requests	Read SERVER_SHOOTOUT.md for the measured Lemonade llamacpp-rocm b1259 profile before planning a new comparison.	The 8-16 parallel advice was measured on 2026-05-05. Lemonade v11.9.0, checked September 19, is available but unqualified here and has not inherited that result.
You are testing prompt-heavy, vLLM, or future server paths	Read BACKEND_CROSSOVER.md and VLLM_BASELINE.md.	Useful for prompt processing, batching, vLLM, and long-context experiments.

Once your first answer has passed its checks, choose a model and an interface for your daily work from the start route. To compare numbers, use the exact commands and evidence links in Reproduce One Headline Result, the AI/search setup summary, headline claim index, and raw evidence under data/raw/.

Headline Evidence

The machine-readable index for these rows is data/headline_claims.csv.

Dates below are measurement dates. A row being from May does not mean it is stale; it means later checks did not produce a stronger replacement headline. The latest controls and current-model rows are documented in CURRENT_MODELS.md, BENCHMARKS.md, PERFORMANCE_NOTES.md, and the raw evidence links below.

Claim	Date	Backend	Model	Result	CSV	Raw	Chart	Notes
Fastest direct 30B-class Qwen MoE route	2026-06-02; b9544 control 2026-06-07	llama.cpp Vulkan/RADV b9467; b9544	Qwen3-30B-A3B-Instruct-2507 IQ4_XS	100.04 tg128 r50, 1416.03 pp512; b9544 control 103.18 tg128 r10	benchmarks	raw r50, b9544 control	n/a	first local direct llama-bench row above 100 t/s; separate general-instruct Qwen route; not the Qwen3-Coder headline or balanced default
Fastest current small-MoE scout	2026-06-05; b9544 control 2026-06-07	llama.cpp Vulkan/RADV 2016bf2; b9544	LFM2.5 8B-A1B Q4_K_M	168.96 tg128, 3414.61 pp512; generation-only 170.02 tg128; b9544 control 176.48 tg128 r10	benchmarks	raw latest/int-dot, b9544 control	n/a	small active-parameter MoE speed/currentness row; not a 30B-class model replacement
Largest current direct GGUF capacity route	2026-07-16	llama.cpp Vulkan/RADV b10034	DeepSeek V4 Flash 284B UD-IQ2_XXS	155.64 pp512 / 13.27 tg128 r3; deterministic answer 9	benchmarks	raw direct pass	n/a	pinned 90.86GB ordinary GGUF; direct load/basic-correctness capacity proof, not a speed or broad quality recommendation
Current 120B-class direct GGUF route	2026-06-05; b9544 control 2026-06-07	llama.cpp Vulkan/RADV 2016bf2; b9544	Nemotron 3 Super 120B-A12B UD-IQ4_XS	18.43 tg128, 294.99 pp512; b9544 control 18.93 tg128 r3	benchmarks	raw latest/int-dot, b9544 control	n/a	direct 120B-class MoE route on one 128GB Strix Halo system; capacity/current-model proof, not a speed headline
Fastest measured short-context coding MoE speed-first quant	2026-06-30	llama.cpp Vulkan/RADV b9851 official release binary	Qwen3-Coder 30B-A3B Q4_K_S	100.99 tg128 r50, 1423.05 pp512	benchmarks	raw r50	n/a	speed-first lower-quality quant; not the balanced UD default; older b9179 strict-clean row remains preserved at 98.51 t/s
Fast balanced short-context coding MoE	2026-05-07	llama.cpp Vulkan/RADV b9049	Qwen3-Coder 30B-A3B UD-Q4_K_XL	96.76 tg128, 1320.52 pp512	max campaign	raw run	n/a	max-performance r20 confirmation; previous b9010 peak was 97.24 t/s
Default Qwen3.6 direct path	2026-05-07	llama.cpp Vulkan/RADV b9049	Qwen3.6 35B-A3B UD-Q4_K_M	62.56 tg128, 1059.45 pp512	benchmarks	raw run	chart	clean latest-stack r20 rerun; rounds to 63 t/s
Fastest measured Qwen3.6 speed-first quant	2026-05-07	llama.cpp Vulkan/RADV b9049	Qwen3.6 35B-A3B Q4_0	81.30 tg128, 1243.51 pp512	max campaign	raw run	n/a	speed-first lower-quality quant; not the default all-round recommendation without a quality sanity check
Current normal Ollama buyer path	2026-07-10	Ollama 0.31.2 system service Vulkan/RADV	Qwen3.6 35B-A3B Q4_K_M	60.57 t/s warm API generation average	benchmarks	raw service run	n/a	normal service-upgrade path; iGPU, vision, service restart, and full-host reboot persistence passed; later controlled local binaries did not reproduce a version-wide slowdown
Experimental Qwen3.6 MTP server path	2026-05-27	llama-server Vulkan/RADV b9360	Qwen3.6 35B-A3B MTP IQ4_XS-Q8nextn	101.16 t/s best local average over six prompts; t16 repeats at 101.15 / 101.10 / 101.06 t/s; 93.29 t/s GMKtec community average on b9235	MTP CSV	local raw, GMKtec raw	n/a	server/speculative result; localweights Q8-next-token-head quant; not the direct llama-bench headline
Experimental Gemma 4 QAT MTP server path	2026-06-12	llama-server Vulkan/RADV ac4cddeb0	Gemma 4 26B-A4B IT QAT UD-Q4_K_XL + Q4_0 MTP head	102.69 t/s cold repeat; 107.42 t/s T3-only repeat; 110.00 t/s best repeat; 73.96 t/s no-spec baseline	MTP CSV	cold raw, T3-only raw, warm raw	n/a	historical server/speculative result with matched QAT MTP head; sampled coding outputs failed and profile is not recommended (see the saved-output review); host-workload sensitive; not direct llama-bench
Experimental CHADROCK ROCmFP4 MTP server path	2026-06-21	llama-server ROCmFPX/RADV helper route	CHADROCK ACE/SABER 35B ROCmFP4	140.40 and 139.93 t/s gen512 high-acceptance repeats; 127.77 t/s gen2048 check	MTP CSV	raw helper repro	n/a	server/speculative result with ciru-ai/ROCmFPX; prompt/acceptance-sensitive; not direct llama-bench
Historical May 80B Qwen-family path	2026-05-16	llama.cpp Vulkan/RADV b9172	Qwen3-Next 80B-A3B UD-Q4_K_XL	59.06 tg128, 751.70 pp512	benchmarks	raw r20	n/a	b9172 improved this 80B MoE path versus the older 54.92 t/s b8933 row
gpt-oss-120b loaded locally	2026-05-07	llama.cpp Vulkan/RADV b9049	gpt-oss-120b MXFP4 split GGUF	55.57 tg128, 726.99 pp512, 293.73 pp65536 r1	max campaign	raw run	n/a	performance evidence only; no model-quality eval; pp65536 is one repeat
User-local Ollama comparator	2026-07-02	Ollama 0.31.1 local binary Vulkan/RADV	Qwen3.6 35B-A3B Q4_K_M	71.82 t/s warm API generation average	benchmarks	raw API run	n/a	9 warm API runs after one cold run; required OLLAMA_IGPU_ENABLE=1; faster than the earlier normal-service check, but a later controlled comparison found no version-wide 0.31.2 regression
Best measured Qwen3.6 server split	2026-05-05	Vulkan/RADV and Lemonade ROCm	Qwen3.6 35B-A3B UD-Q4_K_M	Vulkan wins 1-4 parallel; Lemonade ROCm wins 8-16	server data	raw sweep	n/a	5 reps per concurrency, 0 errors
HIP/Vulkan workload split	2026-05-07	Vulkan/RADV and ROCm/HIP from b9049 source	Qwen3.6 35B-A3B, Qwen3-Coder 30B-A3B	HIP won pp16384; Vulkan won tg128 on both local Qwen rows	max campaign	same-source matrix	n/a	HIP binary reports unknown build id due container git safe-directory, but source checkout was b9049
Best measured Qwen3-Coder local API point	2026-05-03	llama-server Vulkan/RADV b9010	Qwen3-Coder 30B-A3B UD-Q4_K_XL	173.16 aggregate t/s at -np 8	multi-user	raw summary	chart	-np 16 regressed
128K filled-context Qwen3.6 decode completed	2026-05-03	llama-server Vulkan/RADV b9010	Qwen3.6 35B-A3B UD-Q4_K_M	32.23 t/s decode after 128K fill, no truncation	filled KV	raw 128K summary	chart	f16 KV, synthetic long prompt
Real documents are slower than synthetic repeated prompts	2026-05-03	llama-server Vulkan/RADV b9010	Qwen3.6 35B-A3B and Qwen3-Next 80B-A3B	real 64K prompt ingest was 24-33% slower; decode barely changed	filled KV	real-corpus summary	chart	avoids overclaiming synthetic prompt speed

Reproduce One Headline Result

Pick the row that matches what you want to verify. The balanced coding row is the best first reproduction target for most users. The direct 100 t/s row is useful if you specifically want to verify the fastest measured 30B-class Qwen speed scout. Keep them separate: they use different model files, quants, builds, and quality tradeoffs.

Balanced Qwen3-Coder Row

AMD_VULKAN_ICD=RADV \
VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/radeon_icd.json \
~/llama-cpp-upstream-2026-05-07/build-vulkan/bin/llama-bench \
  -m ~/models/Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf \
  -fa 1 -ngl 999 -mmp 0 -p 0 -n 128 -r 20 -o csv

Measured local result: 96.76 tg128 in the max-performance b9049 campaign: raw CSV. This is the practical balanced Qwen3-Coder row. The fastest first-party Qwen3-Coder row is now 100.99 tg128 with Q4_K_S on the official b9851 Vulkan release binary, but that is a speed-first quant rather than the balanced default: raw b9851 r50. The older b9179 strict-clean 98.51 t/s row remains preserved for historical host-state context: raw b9179 r50.

Direct 100 t/s Qwen3 30B Speed Scout

Use this only if you have the exact Qwen3-30B-A3B-Instruct-2507 IQ4_XS-3.63bpw GGUF and a comparable b9467 Vulkan/RADV build. Adjust the binary and model paths to your checkout; the flags, model hash, build, and raw CSV are the reproducibility anchors. This is a direct llama-bench result, but it is not Qwen3-Coder and not the bal

README 内容过长，已截取前 64 KB。完整内容请前往 GitHub 仓库查看。