kaggle-tpu-lab
Run open models on Kaggle’s free TPU v5e-8 and get a public endpoint that speaks the
OpenAI and Anthropic APIs. Point Claude Code, Codex CLI, opencode or anything else at it.
No GPU, no cloud bill, about twenty minutes from pressing Run to a URL.
Each model has its own folder with a run-all Kaggle notebook, the kernel script behind
it, and a write-up of how it works and what we measured.
| Model |
Weights on the TPU |
Context |
One stream |
Many streams |
Prefill |
Run → URL |
Engine |
|
| Qwen3.8-27B |
bf16, no quantization |
262k |
~130 tok/s |
~540 tok/s at 8 |
10,300 tok/s |
~22 min |
vllm-tpu + one patch |
notebook |
| GLM-5.3-Flash (320B MoE) |
3-bit experts, int8 rest |
262k |
~165 tok/s |
~170 tok/s at 3 |
~1,800 tok/s |
6–13 min |
our own JAX engine |
notebook |
Numbers are measured on the shipped configuration; the folder READMEs say how. Qwen runs on
vllm-tpu with one patch. GLM-5.3-Flash runs on an engine we wrote in JAX for it; as far as we
know it is the first to run that model on a TPU.
What you need
A Kaggle account with TPU access (verify it under Settings: a phone number, and for some accounts an identity check through Persona before TPU sessions are handed out) and its free quota,
around 20 TPU hours a week. Nothing to install for the notebook route. For the terminal
route, Python 3.9+ and the Kaggle CLI.
How a session works
The notebook attaches public datasets holding the weights (and, where it helps, a
pre-built compile cache), builds the engine across the eight chips, opens a Cloudflare
tunnel and prints the URL and an API key. The last cell is the server: leave it running.
A keepalive holds the session up to Kaggle’s limit, a little under nine hours, after
which you run it again and get a new URL.
Wire a coding agent with one line, for example Claude Code:
ANTHROPIC_BASE_URL=<url> ANTHROPIC_AUTH_TOKEN=<key> ANTHROPIC_MODEL=<model> claude
Each folder README has the exact lines for Claude Code, Codex CLI and opencode.
From a terminal
git clone https://github.com/ARahim3/kaggle-tpu-lab
cd kaggle-tpu-lab
python launch.py serve # Qwen3.8-27B today; --model picks another recipe once there is one
launch.py pushes the kernel with the Kaggle CLI and follows its progress; status
and stop do what they say.
On a Google Cloud TPU VM
The GLM-5.3-Flash recipe also runs on a rented TPU VM, the same machine as Kaggle’s (v5litepod-8) or a v6e-8,
with no queue and no nine-hour cap: gcp/ has the guide and the three scripts (environment, datasets, run).
Adding a model
One folder at the top level, named after the model: README.md with the numbers and the
how, kernel/ with the serving script, notebook/ with the run-all notebook generated
from it, plus whatever the recipe needs (a patch, an engine). The launcher and the
notebook share the same config block, so a setting changed in one place means the same
thing in the other.
License
The code here is MIT. Model weights keep their own licenses; each folder says which.
评论