Project_Chronos

Project Chronos (Experimental)

A storage-aware MoE stack built for SSD+DRAM hybrid inference, with a full six-stage training pipeline.

中文文档

Why reactive MoE offload breaks down

Mainstream MoE models such as Mixtral, DeepSeek-MoE, and Qwen-MoE make routing decisions at decode time, token by token. The model discovers which experts it needs only when the next token is already on the critical path. If those experts are not resident in VRAM, generation stalls while weights are moved from SSD to RAM to VRAM.

That is not a tuning issue. It is an architectural mismatch.

Most offload runtimes assume the model was originally designed for full VRAM residency, then try to patch storage pressure afterward. On consumer hardware, that usually means the decode loop ends up paying the IO bill over and over again.

Chronos in one line

Move IO into prefill. Move synchronization down to the expert-event level.

Two things matter here:

1. Prefill-time loading: PrefillScheduler and IntentClassifier bulk-load predicted experts before the first decode token.
2. Per-expert event sync (M3): promote_to_vram(blocking=False) records a torch.cuda.Event on the _h2d_stream, and the compute stream waits only on the experts it actually needs. No more stream.synchronize() for the whole system.

Under a simulated 30 ms SSD delay, the new path keeps 35 ms+ per token of pipeline slack compared with the older blocking path.

v3 kernel: recurrent-depth MoE (0.3.0)

v3 reuses a few physical layers many times. A prelude runs once, each shared recurrent group loops recurrent_depth times with a contractive update h ← h + dt·(block(h + e_loop) − h), and a coda runs once. The 50M demo has 6 physical layers that execute as 18 virtual layers (prelude 1, 4 recurrent layers × depth 4, coda 1).

What changed from v2

per_iteration (one KV slot per virtual layer) is kept as an ablation. Old v2 checkpoints load as first_only; their logits match a stored golden tensor within 1e-5.

Self-speculative decoding (no extra parameters). The same model drafts γ tokens at a shallow depth and verifies them with one full-depth batched forward pass. Greedy output is token-for-token identical to full-depth greedy. Sampling uses rejection sampling, which preserves the full-depth distribution. After a rejection, the cache is truncated: MLA slices c_kv, and the sliding-window cache keeps window + γ_max entries. Torch (chronos/runtime/speculative.py) and MLX (chronos/mlx/speculative.py, ChronosMLXInferenceEngine.generate(speculative=True)) share the algorithm. Tests cover greedy identity, the rejection distribution and rollback against a fresh prefill on both backends. The Web UI Inference tab has a switch for it, with draft depth and γ.

For the 50M demo it is exact but not faster on any runtime we measured (pretrain step-13,996 checkpoint, greedy, batch 1, FLORES+ dev en→zh, draft depth 1, γ 4; reports in reports/speculative/):

At this size a full-depth step costs about as much as a batched verify pass. With lazy experts, drafts and verify passes also route to different experts, so speculative decoding read 1.4–1.6× more expert pages from SSD. Greedy outputs were identical in CPU fp32 and Torch MPS fp32. In GPU bf16, near-tie tokens flipped in 2 of 24 sentences. The switch therefore stays off by default.

Precision chain. One compute dtype runs end to end. FP32 is used only where it pays: RMSNorm statistics, SDPA softmax, RoPE tables, router softmax, LM-head logits and optimizer master weights. The MLX↔Torch bf16 bridge is bit-exact. auto picks bf16 on MLX GPU and CUDA, and fp32 on CPU and MPS (MPS bf16 measured slower; see below).

Tooling added in v3

chronos.model.factory.fit_config searches for a configuration that matches total and active parameter targets.

Status of the 50M en→zh demo (in progress).

• Model: 49.69M total / 12.09M active parameters, 36 experts, MLA latent 128, SWA window 128, 16k self-built vocabulary.
• Run: a 72-hour MLX bf16 run (pretrain, then SFT) started on 2026-11-05.
• Interim measurement: valid CE 2.477 at pretrain step 13,996 of about 24.7k planned (it was 3.012 at step 1,656). That checkpoint is packaged as chronos-enzh-50m-pretrain-s13996 (see Model bundle).
• BLEU/chrF on FLORES+ and WMT22, and the ablations (attention mode, depth, lazy-expert budget, speculative γ), will be published after the run finishes. No quality numbers are claimed before then.

Inference and memory options (all off by default). Each option is a config field and a chronos serve / Inference-tab override. Tests compare each one with its reference path on Torch and MLX.

Web UI: the Tokenizer tab (extract → train → evaluate) and the Serve tab (start/stop chronos serve; the API key goes through the environment, never argv).

Three-tier storage (M1: cluster-aware safetensors)

cluster_manifest.json and .ctsr files are produced offline by Louvain clustering. At runtime, Chronos uses safetensors.safe_open(...).mmap to bring an entire expert cluster into RAM in one shot, turning random reads into mostly sequential reads.

Cache misses degrade gracefully instead of stalling

Even the worst case does not hard-stop generation:

# Pure tensor math: no Python branch, no graph break under torch.compile
output = avail[i] * expert_output + (1.0 - avail[i]) * shared_expert_output

The shared expert is always resident, so generation continues while the missing expert finishes loading in the background. For exact lazy/offload comparison modes, Chronos synchronously materializes only the selected missing expert and evicts low-LRU experts to stay inside the resident budget; it does not silently full-load all experts. Quality degrades smoothly only when fallback mode is explicitly enabled.

Dual-layer routing with supervised lookahead (M2)

Before M2, the lookahead head was just a head with no real supervision. M2 adds a proper soft-target objective:

L_{\mathrm{lookahead}}
= \frac{1}{|K_{\mathrm{valid}}|}
\sum_{k \in K_{\mathrm{valid}}}
\mathbb{E}_{b,t}
\left[
  - \sum_{e=1}^{E}
  \mathrm{sg}(P_{b,t+k,e}) \log Q_{b,t,e}^{(k)}
\right]

That turns the lookahead router into an actual predictor of future routing, instead of a best-effort heuristic.

Full training stack: Stage 1 -> Stage 6

Each stage has its own entry script, and every one of them inherits the shared Chronos loss mixer: balance loss, temporal locality loss, lookahead loss, and, for alignment stages, a router KL anchor that keeps RL/DPO updates from destroying cache locality.

Training dtype and resource policy:

• --dtype auto is the default. One compute dtype covers weights, activations, KV cache and expert pages; FP32 is kept only inside fused kernels (SDPA softmax, RMSNorm statistics), for RoPE tables, router softmax, loss logits and the optimizer’s FP32 masters/moments. auto resolves to BF16 on MLX GPU, BF16 on CUDA (FP16 if unsupported, with loss scaling), and FP32 on CPU and MPS. On MPS, native BF16 measured slower than FP32 (Chronos 50M on an M5 Max with torch 2.10: 0.749 vs 0.619 s per training step, 49.1 vs 37.3 ms per decoded token). Pass --dtype bfloat16 explicitly to trade speed for half the weight and KV memory.
• CPU training configures PyTorch to use physical cores by default. Override with --cpu_threads or --cpu_budget_percent.
• On macOS, MPS/MLX training forces DataLoader workers to 0 by default to avoid Metal command-buffer crashes from multiprocessing. CPU/CUDA still use worker processes; advanced users can override the guard with CHRONOS_ALLOW_METAL_DATALOADER_WORKERS=1.
• Native MLX training pushes UI logs, scalar readouts, and chart points every log_interval steps, and Web UI Stop is checked at each batch boundary.
• The Web UI writes a warning-only <checkpoint>.verify.json after each stage. It checks no-mask vs all-available MoE parity and, on Apple Silicon, MLX prefill logits against the PyTorch CPU baseline.

Native MLX training is a separate Apple Silicon backend, not torch.to("mlx"). The six-stage MLX trainer mirrors the PyTorch loss stack in chronos.mlx.*: masked CE, DPO/ORPO/GRPO/distillation losses, load balance, temporal locality, lookahead soft-target supervision, lookahead top-k hit loss, and router KL anchor. Numerically sensitive pieces run in FP32 even when model weights use BF16/FP16, which is why auto prefers BF16 over FP16 on MLX GPU: FP16 has too little exponent range for router softmax, CE, and Adam moments on small unstable training runs.

The MLX Web UI trainer reports the same fields as CPU/MPS training: step/loss/steps-per-second/ETA, checkpoint-save events, stop events, and verify results. Stop is cooperative at batch boundaries and save intervals write a valid .pth plus sibling .config.json, so MLX stages can resume or feed the PyTorch/export pipeline.

The full six-stage comparison harness lives in tools/compare_minimind_chronos_v3.py.

Backend dispatch (M5)

from chronos.backend import BackendDispatcher

d = BackendDispatcher()
d.available()   # ['mlx', 'mps', 'cpu'] on Apple Silicon
                # ['cuda', 'cpu']        on NVIDIA hosts
d.select()      # choose the best available backend automatically
d.describe()    # human-readable capability summary

• First-class backends for training and inference: cpu, mps, cuda, mlx
• Inference-only / experimental: vulkan when PyTorch was custom-built with USE_VULKAN=ON
• Third-party extension hook: opencl, via chronos/backend/ext/opencl.py:PROBE()
• Apple Silicon policy: inference auto still prefers MLX; training keeps MLX on the native chronos.mlx.* path instead of calling torch.model.to("mlx").

Honest note: upstream PyTorch does not ship a real OpenCL backend, and Vulkan support is still niche. Chronos provides a dispatcher seam so external integrations can plug in cleanly without touching core code.

MLX lazy/offload runtime

MLX uses Apple unified memory, so Chronos treats “VRAM” and “RAM” as logical tiers rather than physical buses. The native lazy runtime still enforces the same offload contract as CUDA/MPS/CPU:

• Hot slots hold only the execution-budget experts materialized as live MLX modules.
• Warm cache holds a bounded prediction buffer loaded from .ctsr safetensors, never a hidden copy of every expert.
• Cold storage is the checkpoint/export reader or per-expert cluster cache. In lazy mode, live experts are replaced by placeholders after cache creation; Chronos does not keep a _saved_live full-expert repair cache.
• Lookahead predictions are queued into the warm cache and only promoted when ready. A true miss synchronously materializes the selected expert only; it does not full-load the model.
• MLX attention dynamically grows RoPE lookup tables past max_position_embeddings, so long prompts plus decode do not fail at token 257/513/etc.

Inference compare/sweep reports real hot/warm counts, resident hit rate, prediction hit rate, sync SSD loads, MLX active/cache/peak memory, process RSS, prefill/decode time, and tokens/sec. A lazy run is not considered offload-ready unless deterministic exact-lazy output matches full-DRAM output and fallback weight stays zero.

Hugging Face and vLLM compatibility (M5)

• ChronosForCausalLM subclasses PreTrainedModel and registers AutoConfig and AutoModelForCausalLM, so loading does not require trust_remote_code:

  from transformers import AutoModelForCausalLM
  
  model = AutoModelForCausalLM.from_pretrained("./out_dir")
  

• chronos.model.hf_io.save_chronos_pretrained and load_chronos_pretrained emit standard model.safetensors + config.json, while also carrying cluster_manifest.json and .ctsr files for expert-cache layout. Roundtrip logit drift is 0.00e+00.

• chronos.serving.register_chronos_with_vllm() registers Chronos with the vLLM ModelRegistry when vLLM is installed. If vLLM is absent, it prints an install hint and exits cleanly. Worker-side mask injection is documented in docs/vllm_integration.md.

Compared with existing offload stacks

Objective

L_{\mathrm{total}} =
L_{\mathrm{base}}
+ \lambda_{\mathrm{bal}} L_{\mathrm{aux}}
+ \lambda_{\mathrm{tmp}} L_{\mathrm{temporal}}
+ \lambda_{\mathrm{LA}} L_{\mathrm{lookahead}}
+ \lambda_{\mathrm{anc}} L_{\mathrm{routerKL}}

L_{\mathrm{aux}} = E \sum_{e=1}^{E} load_e \cdot \overline{p}_e

L_{\mathrm{temporal}} =
\mathbb{E}_{b,t}
\left[
  \left\| P_{b,t,:} - P_{b,t-1,:} \right\|_2^2
\right]

L_{\mathrm{routerKL}} =
D_{\mathrm{KL}}
\left(
  \pi_{\theta}^{\mathrm{router}}
  \|
  \pi_{\mathrm{ref}}^{\mathrm{router}}
\right)

• L_base: stage-specific objective (CE, DPO, ORPO, GRPO, or distillation).
• L_aux: the unscaled MoE load-balance auxiliary term; Chronos applies lambda_bal once in chronos_loss_term.
• L_temporal: encourages adjacent tokens to reuse similar expert distributions.
• L_lookahead: soft-target cross entropy from the real future router distribution to the lookahead prediction. sg(...) means stop-gradient.
• L_routerKL: keeps alignment-stage updates from destroying the routing layout captured at stage start.

All lambda terms are searchable with Optuna TPE, together with structural hyperparameters such as hidden_size, num_experts, and kv_latent_dim.

Released model: chronos-enzh-50m-sft

models/chronos-enzh-50m-sft/ packages a 49.7M-parameter Chronos v3 model (12.1M active per token) trained for 69 hours on one Apple M5 Max: configuration, tokenizer, trained prompt, decoding settings, a driver (translate.py) and the evaluation behind every number. The weights (99.5 MB, bfloat16 safetensors) are attached to the release enzh-50m-sft-v1.

• Boundary: English → Chinese only, one sentence (≤ 128 tokens) at a time. Chinese → English is disabled: English-only fine-tuning erased it (FLORES+ BLEU 0.00 after SFT). The driver refuses input outside the boundary.
• Measured (MLX bf16, greedy): FLORES+ devtest BLEU 33.48 / chrF 29.52; WMT22 BLEU 32.60 / chrF 28.83; WMT24++ BLEU 26.89 / chrF 24.43 (split into sentences).
• Research use only. The training corpora’s licences are unknown; see the folder’s NOTICE.

cd models/chronos-enzh-50m-sft
pip install --pre "Project_Chronos[mlx]==0.3.2a0"   # or "Project_Chronos==0.3.2a0" without MLX
python translate.py --download && python translate.py "The meeting was moved to Friday."

Installation

pip install Project_Chronos

Or from source:

git clone https://github.com/FonaTech/Project_Chronos
cd Project_Chronos
pip install -e ".[dev]"

MLX (Apple Silicon):

pip install "Project_Chronos[mlx]"

Optional extras (versions pinned):

pip install "Project_Chronos[serve]"      # chronos serve / chronos monitor (fastapi, uvicorn)
pip install "Project_Chronos[translate]"  # corpus detokenization + BLEU/chrF probes (sacremoses, sacrebleu)

vLLM serving (optional, Linux + CUDA only):

pip install vllm

minimind dependency: Project Chronos uses minimind as its MoE kernel. If it is not found locally, Chronos clones it automatically into ~/.cache/chronos/minimind-master/ on first import (this needs git and network access). Set CHRONOS_MINIMIND_DIR=/path/to/minimind to use a local copy instead; model bundles ship one under third_party/minimind and their launcher sets the variable. minimind is licensed under Apache-2.0. See THIRD_PARTY_NOTICES.md for attribution details.

Requirements: Python 3.10+, PyTorch 2.6+

Quick start

Web UI (12 tabs, 4 languages)

chronos-ui
# or
python chronos_app.py

Tabs included:

• Config: v3 presets, the auto planner (below) and a live parameter / memory estimator
• Data for dataset scanning and conversion
• Tokenizer: extract text from datasets, train the Chronos BPE, evaluate and save
• Train with its own data_path; applies the selected stage’s overrides
• Training Monitor (read-only view of runs/; see below)
• 6-Stage Pipeline with per-stage dataset paths
• Inference
• Export for FP16/Q8_0 safetensors and GGUF deployment artifacts
• Serve: start and stop chronos serve and view its log
• Benchmark with Markdown table + comparison plots
• Auto-Tune with persistent logs and one-click Apply Best -> Config
• IO Monitor

At a 1280 px wide window, Gradio moves the last tabs (Auto-Tune, IO Monitor) into its “…” overflow menu.

Presets and the auto planner

The Config tab ships five v3 presets. Each one is generated by the planner and stored in ui/presets_v3.json; regenerate the file with python -m ui.presets. Parameter counts are measured with a meta-device model; peak memory is an estimate.

The old preset names (Recommended-CN, Tiny/Small/Medium/Large) still load; they map to the closest v3 tier.

Auto planner (Config tab → “Auto plan”). Inputs:

• target total parameters, and whether to use MoE;
• the expert count, plus either the number of active experts or an active-parameter target;
• a memory-saving level: none, balanced, aggressive or extreme;
• optional dataset paths, RAM, token budget, vocabulary size and sequence length.

chronos.model.planner.plan_training then does the following:

1. Profiles the datasets. It reads the shard manifest or samples the files, classifies records as pretrain, SFT, preference or RL, and reads the tokenizer report. The vocabulary size, sequence length (p95 / cap), token count and tokenizer path come from the data when available.
2. Searches the architecture. It looks for the hidden size, layer count, expert width and GQA heads whose meta-device count is closest to the target.
3. Sets LR and batch size from the DeepSeek LLM compute-optimal fits (η = 0.3118·C^−0.125, B = 0.2920·C^0.3271), capped for wide models.
4. Fits memory. It picks micro-batch, accumulation, int8 KV, router scope, activation checkpointing, optimizer paging and an expert RAM budget for the memory level.
5. Writes per-stage values. Pretrain, SFT, DPO, ORPO, GRPO and distill each get their own learning rate, batch, epochs, save interval and stage-specific terms (β, temperature, α).

The stage table is editable; the Train tab merges the selected stage’s row over the shared values, and the pretrain row stays in sync with the shared widgets. “Save plan” writes the config, stage values and report to JSON. Every number in the report is labelled measured or estimated.

Built-in i18n: zh-Hans, zh-Hant, en, ja

Training monitor (console tab + standalone page/API)

A chronos longrun run directory holds run_spec.json, metrics.jsonl, status.json (heartbeat), runtime.json (precision policy, device, versions, code hash), parameters.jsonl (per-tensor weight/grad/update statistics and per-virtual-layer routing), samples.jsonl, inspections/ and checkpoints. The console’s Training Monitor tab and a standalone page both read these files and never touch the training device. Runs started before telemetry existed still show progress; their status is derived from metrics.jsonl and the process table.

chronos monitor --runs-dir runs                    # http://127.0.0.1:7870/
chronos serve --models-dir exports --monitor-runs-dir runs   # same page at /monitor/

JSON API (read-only): GET /api/state?run=&history=1, /api/metrics?run=&phase=&since=, /api/parameters, /api/tensors, /api/tensor?name=, /api/log?bytes=, /api/runs.

Security: the default bind is 127.0.0.1 with no authentication. Any other address requires --allow-remote and --api-key (or CHRONOS_MONITOR_KEY). Browsers sign in once and get an HttpOnly, SameSite=Strict session cookie; scripts send Authorization: Bearer <key>. Failed logins are rate-limited. Off loopback, add --ssl-certfile/--ssl-keyfile or a TLS reverse proxy, otherwise the key crosses the network in clear text.

Ollama / OpenAI protocol server

chronos serve runs the native Chronos runtime behind the Ollama and OpenAI HTTP APIs. The kernel is not converted to another architecture, so recurrent depth, SSD-lazy experts with lookahead prefetch, compressed KV and self-speculative decoding stay active.

pip install "Project_Chronos[serve]"
chronos serve --models-dir ./exports --port 11435        # 11434 would collide with a running Ollama
OLLAMA_HOST=127.0.0.1:11435 ollama list
OLLAMA_HOST=127.0.0.1:11435 ollama run <model-name> "Hello"

• Endpoints: /api/version, /api/tags, /api/show, /api/ps, /api/pull, /api/generate and /api/chat (streaming NDJSON), /v1/models, /v1/chat/completions (SSE).
• Models are <name>.pth + <name>.config.json pairs found under --models-dir.
• Chronos options go in the request options: chronos_speculative, chronos_draft_depth, chronos_gamma, chronos_miss_policy.
• Stock ollama / llama.cpp cannot execute Chronos GGUF files (MLA+SWA hybrid layers and the contractive recurrence have no stock equivalent); use this server instead.
• Security: binds 127.0.0.1 without authentication by default. Other addresses require --allow-remote and --api-key; clients then send Authorization: Bearer <key>.

Model bundle (ready-to-run pretrained model)

chronos bundle build packages one exported checkpoint of a chronos longrun run into a folder that works right after unpacking. Building only reads the run directory.

chronos bundle build --run-dir runs/enzh_50m_v1 --which best --wheel dist/project_chronos-0.3.0-py3-none-any.whl --pack

On the recipient side:

bin/setup.sh                                              # venv + wheel + `bundle check`
bin/chronos bundle translate "The weather is very nice today."
bin/chronos bundle translate --backend mlx --speculative --file examples/translate_en.txt
bin/chronos bundle eval --split devtest -n 200
bin/chronos bundle prepare-data my_pairs.jsonl --out work/data      # FLORES+ is removed from your data
bin/chronos bundle finetune --data work/data --run-dir work/ft --tier quick --mode sft
bin/chronos bundle serve -- --port 11435

--speculative needs --repetition-penalty 1.0 (its default), because the penalty changes the target distribution at every position. finetune writes a chronos longrun spec with init_checkpoint set to the bundle weights and refuses a run directory that already holds a run. Bundles built from data with unknown licences (WMT19, OPUS-100) say in NOTICE.md that the weights are for research only.

Deployment export

chronos export \
    --model_path ./out/sft_384_moe.pth \
    --output_dir ./exports/sft_384 \
    --formats fp16-safetensors q8_0-safetensors fp16-gguf q8_0-gguf

Exports include config.json, chronos_export_manifest.json, and Chronos metadata for MoE top-k, shared fallback experts, lookahead router, hybrid attention, and optional expert-cache layout. Chronos can load exported safetensors/GGUF artifacts through its native lazy expert loader.

Compatibility note: the GGUF files use general.architecture=chronos. Stock Ollama/llama.cpp builds need a Chronos architecture adapter to execute them correctly; Chronos is not a LLaMA tensor-layout clone.

Stage 1: pretrain

python train_chronos.py \
    --data_path ./tests/fixtures/tiny_pretrain.jsonl \
    --hidden_size 256 --num_hidden_layers 4 --num_experts 4 \
    --epochs 1 --device cpu --save_dir ./out

Stage 2-5: alignment chain

python train_chronos_sft.py  --data_path ./tests/fixtures/tiny_sft.jsonl  --from_weight chronos --save_dir ./out --device cpu
python train_chronos_dpo.py  --data_path ./tests/fixtures/tiny_dpo.jsonl  --from_weight sft     --save_dir ./out --device cpu
python train_chronos_orpo.py --data_path ./tests/fixtures/tiny_dpo.jsonl  --from_weight sft     --save_dir ./out --device cpu
python train_chronos_grpo.py --data_path ./tests/fixtures/tiny_grpo.jsonl --from_weight orpo    --save_dir ./out --device cpu \
    --reward toy   # or lm:/path/to/reward-model

Stage 6: distillation

python train_chronos_distill.py \
    --data_path ./tests/fixtures/tiny_sft.jsonl \
    --teacher_path ./out/sft_192_moe.pth \
    --from_weight grpo --save_dir ./out --device cpu \
    --alpha 0.7 --temperature 4.0

Checkpoint and offload diagnostics

Every new .pth checkpoint writes a sibling *.config.json with the MoE topology that cannot be recovered from tensor shapes, including num_experts_per_tok. Use the diagnostic command to verify chat-template generation, no-mask vs all-available masked drift, cold shared fallback, LookaheadRouter prediction quality, and SSD/RAM/VRAM offload stats.

python diagnose_checkpoint.py \
    --model_path ./out/sft_384_moe.pth \
    --config_path ./chronos_config.json \
    --sft_data ./Dataset/sft_t2t.jsonl \
    --mlx_parity \
    --device cpu

# or through the unified CLI:
chronos diagnose --model_path ./out/sft_384_moe.pth --config_path ./chronos_config.json

For backend speed and dtype sanity checks:

python benchmark_training_backends.py --backends cpu mps mlx --dtypes auto bfloat16 float16 --steps 2

End-to-end comparison (minimind vs Chronos)

python tools/compare_minimind_chronos_v3.py \
    --pretrain_steps 150 --align_steps 30 --distill_steps 30 \
    --simulated_ssd_ms 30 --device cpu \
    --output results/compare_results_v3.json

This emits per-stage loss, HF roundtrip logit delta, tokens/sec, active-expert ratio, resident-expert bytes, M3 pipeline slack, and the backend inventory on the current host.

Cluster-pack expert storage for sequential SSD reads

from chronos.io.cluster_layout import (
    collect_activation_log,
    build_cooccurrence_matrix,
    try_louvain_clustering,
    repack_expert_weights_safetensors,
)

log = collect_activation_log(model, calib_loader, "cpu", max_batches=50)
clusters = try_louvain_clustering(build_cooccurrence_matrix(log, num_experts))
repack_expert_weights_safetensors(model, clusters, "./expert_cache_clustered")

Search lambdas and structure hyperparameters automatically

from chronos.tuning.chronos_auto_tuner import ChronosAutoTuner, ChronosSearchSpaceConfig

tuner = ChronosAutoTuner()
tuner.start(
    model_id="./out/chronos_256_moe.pth",
    dataset_path="./dataset/train.jsonl",
    search_space=ChronosSearchSpaceConfig(
        tune_lambda_balance=True,
        tune_lambda_temporal=True,
        tune_lambda_lookahead=True,
        tune_lookahead_steps=True,
        tune_hidden_size=True,
        tune_num_experts=True,
        tune_num_shared_experts=True,
        tune_kv_latent_dim=True,
    ),
    n_trials=20,
)

Project layout

Project_Chronos/
├── chronos/
│   ├── deps.py                    # Auto-download minimind if missing
│   ├── __init__.py                # AutoConfig / AutoModelForCausalLM registration
│   ├── model/
│   │   ├── config.py              # ChronosConfig (v3 fields, v2 migration)
│   │   ├── recurrent.py           # Loop/group embeddings, contractive step, tensor-only ACT
│   │   ├── precision_ops.py       # RMSNorm/SDPA/RoPE with one compute dtype
│   │   ├── factory.py             # Parameter-budget config search
│   │   ├── hybrid_attention.py    # MLAAttention + SlidingWindowAttention
│   │   ├── lookahead_router.py    # Per-token lookahead predictor
│   │   ├── moe_chronos.py         # ChronosMOEFeedForward + shared experts + soft gating
│   │   ├── model_chronos.py       # ChronosForCausalLM
│   │   ├── temporal_loss.py       # Temporal locality + lookahead losses
│   │   └── hf_io.py               # save/load_chronos_pretrained + HF registration
│   ├── io/
│   │   ├── expert_store.py        # Three-tier storage + per-expert events
│   │   ├── async_prefetcher.py    # Async prefetch engine
│   │   ├── storage.py             # ClusterStorage: .ctsr safetensors + manifest
│   │   ├── cluster_layout.py      # Co-occurrence clustering + repacking
│   │   └── io_simulator.py        # CHRONOS_SIM_SSD_MS test hook
│   ├── router/
│   │   ├── intent_classifier.py   # Prompt-level expert predictor
│   │   ├── expert_predictor.py    # IntentVector -> ExpertSet
│   │   └── prefill_scheduler.py   # Prefill-time expert preloader
│   ├── mlx/
│   │   ├── attention.py / moe.py / model.py / expert_store.py / inference.py
│   │   ├── moe_kernels.py         # Sorted gather_mm dispatch (mlx >= 0.32.4)
│   │   ├── speculative.py         # MLX self-speculative decoding (Torch parity)
│   │   └── training/              # longrun.py, telemetry.py, recipes/
│   ├── runtime/
│   │   ├── speculative.py         # Recurrent-depth self-speculative decoding
│   │   ├── cache_manager.py       # prefetch_for_next_step / ensure_resident
│   │   ├── inference_engine.py    # End-to-end inference engine
│   │   └── metrics.py             # MetricsBus for the IO Monitor
│   ├── trainer/
│   │   ├── loss_mixin.py          # chronos_loss_term + router_kl_anchor
│   │   ├── chronos_trainer.py     # Pretrain
│   │   ├── sft_trainer.py         # Stage 2
│   │   ├── dpo_trainer.py         # Stage 3
│   │   ├── orpo_trainer.py        # Stage 4
│   │   ├── grpo_trainer.py        # Stage 5
│   │   ├── distill_trainer.py     # Stage 6
│   │   └── reward.py              # ToyReward / LMRewardModel / build_reward_fn
│   ├── tuning/
│   │   └── chronos_auto_tuner.py  # Optuna lambda + architecture search
│   ├── eval/
│   │   ├── io_profiler.py         # Lookahead validation
│   │   └── benchmark.py           # End-to-end benchmarking
│   ├── data/
│   │   ├── flexible_dataset.py    # Flexible JSONL dataset loader
│   │   └── corpus/                # download / clean / dedup / leak filter / shards
│   ├── tokenizer/                 # Chronos BPE builder (extract, normalize, train, evaluate)
│   ├── monitor/                   # Read-only run monitor: snapshot, API server, static page
│   ├── backend/
│   │   ├── __init__.py            # BackendDispatcher (cpu/mps/cuda/mlx)
│   │   ├── dispatcher.py          # Capability probing + priority logic
│   │   └── ext/opencl.py          # Third-party OpenCL extension hook
│   ├── _backend_legacy.py         # Backward-compatible older APIs
│   ├── bundle/                    # chronos bundle: build/check/translate/finetune
│   ├── serving/
│   │   ├── ollama_server.py       # chronos serve: Ollama/OpenAI protocol layer
│   │   ├── runtime.py             # Model discovery, lazy experts, speculative generation
│   │   └── vllm_adapter.py        # Optional vLLM registration
│   └── cli.py                     # Unified CLI
├── ui/                            # Gradio Web UI (zh-Hans / zh-Hant / en / ja)
│   ├── i18n.py
│   ├── estimator.py               # Live parameter / memory estimator
│   └── tabs/
│       ├── config_tab.py          # Config + Designer merged together
│       ├── data_tab.py
│       ├── train_tab.py           # Owns data_path
│       ├── monitor_tab.py         # Training Monitor (shares chronos.monitor)
│       ├── pipeline_tab.py        # Per-stage datasets across all 6 stages
│       ├── inference_tab.py
│       ├── benchmark_tab.py       # Markdown table + gr.BarPlot
│       ├── autotune_tab.py        # Persistent logs + Apply Best -> Config
│       └── iomon_tab.py           # MetricsBus dashboard
├── chronos_app.py                 # Web UI entry
├── train_chronos.py               # Stage 1 entry
├── train_chronos_sft.py           # Stage 2 entry
├── train_chronos_dpo.py           # Stage 3 entry
├── train_chronos_orpo.py          # Stage 4 entry
├── train_chronos_grpo.py          # Stage 5 entry
├── train_chronos_distill.py       # Stage 6 entry
├── tools/
│   ├── compare_minimind_chronos.py
│

提示：代码过长已自动截断

Roadmap

Citation

@misc{chronos2026,
  title  = {Project Chronos: Prefill-Time Expert Loading and Dual-Layer Routing
             for Zero-Stall On-Device MoE Inference},
  author = {Fona and Project Chronos Contributors},
  year   = {2026},
  url    = {https://github.com/FonaTech/Project_Chronos}
}

Third-party attribution

Project Chronos builds on jingyaogong’s minimind, licensed under Apache-2.0. Full attribution lives in THIRD_PARTY_NOTICES.md.

License

Apache 2.0 - see LICENSE