AI Janitor / Learn
How a model computes
A model is a fixed list of sums and the steps between them. Four ideas, then open any architecture.
Sums
A model is a fixed list of sums: weights times inputs, added up. A Qwen3-ASR decode step runs 169 of them, and one of its rows adds 2,048 products.
Phases
The work comes in phases: read the input once, then produce one token at a time. Inside a phase, the same layer repeats, 24 or 28 times.
Order
Some steps must wait: a layer needs the previous one; softmax needs its whole sum. Inside one sum the order is free with exact math. In fp32 every add rounds, so another order only moves the last bits.
FIXED layer 1 ──▶ layer 2 ──▶ … ──▶ layer 28
sum ──▶ ▮ softmax (waits for the whole sum)
FREE ◇── x₁·w₁ ─┐
── x₂·w₂ ─┼─◆ Σ any order, same exact answer
── … ─┘Open one
Every architecture below is drawn that way: numbered order, ◇ any-order groups, ▮ steps that wait for all, ↻ repeats. Click any box for its numbers.
Architectures
| architecture | family | phases | boxes |
|---|---|---|---|
| Chatterbox TTST3 Llama token LM + S3Gen flow-matching mel decoder + HiFT vocoder | composite | 4 | 131 |
| DeltaNet layerlinear attention with the delta rule, no decay gate | linear attention | 2 | 42 |
| FastConformer encoder + Token-and-Duration Transducer decoderParakeet TDT | transducer | 2 | 84 |
| Gated DeltaNet layerQwen3-Next / Qwen3.5 linear attention | linear attention | 2 | 66 |
| Mamba blockselective state-space model, v1 | state-space | 2 | 52 |
| Mamba-2 blockstate space duality, multi-head SSM | state-space | 2 | 44 |
| Mixture-of-Experts FFNrouter + top-k SwiGLU experts, Qwen3-MoE style | mixture-of-experts | 2 | 32 |
| Qwen3-ASRconv-subsampled audio transformer encoder + adapter + Qwen3 decoder | composite | 5 | 181 |
| RWKV-7 "Goose" blockgeneralized delta rule, time-mix + channel-mix | recurrent | 2 | 96 |
| SAN-M encodermemory-equipped self-attention, DFSMN) + CTC head (SenseVoice | speech-encoder | 1 | 42 |
| Decoder-only transformerLlama / Qwen3 family: RMSNorm, GQA attention + RoPE, SwiGLU FFN | attention | 2 | 108 |
| Encoder transformerbidirectional attention, LayerNorm, GELU MLP | attention | 1 | 36 |
Data: model-precision-map · data/architectures/*.json → flow/data/*.json (built once by scripts/flowbuild.py). Theme: aijanitor.si (site/index.html, docs/whole-site-motion-plan.md).