Feasibility measurements on real fleet hardware. The rule these follow: an estimate is scaffolding, the measurement is the number — and a failure ledger is part of the result, not something to edit out.
The question: can Google's 25.2B-parameter diffusion language model run on a $250 edge computer with 8GB of unified memory? Its smallest quantized form is 16.8GB; the stated requirement is ~18GB of VRAM. The presumptive answer was no. The exploration was delegated whole — a one-paragraph request, executed autonomously by Claude (research, system pause, six diagnosed failures, a one-line llama.cpp patch, the measurement, restoration, and this page).
| metric | measured |
|---|---|
| end-to-end throughput | 0.48 tok/s (256-token block in 528.7s) |
| denoising steps | 21 of 48 max (entropy early-stop) |
| NVMe traffic | 296GB total — the model streamed past itself ~17× |
| memory | RSS pinned at 4.18GB by a systemd cgroup; box stayed healthy |
| bottleneck | ~92% of step time is expert streaming from disk |
Not interactive — 30× too slow for conversation, and because the bottleneck is disk, no GPU offload rescues it. But "doesn't fit" turned out to be the wrong frame: the honest answer is a price, not a wall. The dense core (~1.6GB) stays hot; the ~15GB expert mass streams. Block diffusion is the adversarial case for this split — it re-touches nearly every expert on every denoising step — and the tiering still carried it to completion.
The prompt was "Once upon a time, on a small computer at the edge of the network,". The model spent its one block thinking, verbatim:
The canvas ended there. A model streaming itself through an edge device chose to write about a lonely edge device discovering something unexpected. The story begins exactly where the experiment ends.
| config | killer |
|---|---|
| CUDA, defaults | GPU pinned-memory allocation failed under the memory cap |
| CPU-only, defaults | weight repacking builds a ~15GB un-evictable copy → reclaim death-loop |
| CUDA, batch flags | tool hard-codes 2,048 tokens of headroom → demands a 2.96GB buffer |
| CPU variants (3×) | same anonymous-memory blowups by other names |
| CUDA + mmap + 1-line headroom patch | ran to completion |
Lessons that transfer to any >RAM model on unified-memory Jetsons: the mmap + CUDA-host path is the only road; GPU allocations live outside Linux cgroups; audit tool-computed batch sizes (at a 262k vocabulary, every batch token costs ~1MB of logit buffer). Full write-up with raw data: DIFFUSIONGEMMA_SPROUT_FEASIBILITY.md — companion perspective on why "too big for the hardware" is usually the wrong frame: SPARSE_ACTIVATION_MEMORY_HIERARCHY.md.