Overview
Colibri shows how a sparse frontier model can run when its full weight set exceeds fast memory: keep dense state resident, stream routed experts from storage, and adapt placement through caches and measured tiers. The series follows token routing, indexed reads, planning, prediction, and parity checks, and concludes that feasibility is real but speed remains hardware-, workload-, and model-path-specific.
Learning outcomes
- Sparse routing lets the full model stay on storage while selected experts move through RAM or VRAM.
- A larger cache helps only when it holds the experts the workload actually selects.
- Indexed reads, coalesced slabs, and overlap can reduce waiting without changing the selected route.
- Correctness claims must name their boundary: bytes, numerical error, tokens, quality, or end-to-end speed.
Keywords
Preview every chapter · Unlock full text, audio, and video from chapter 4
PlansFAQ
How can Colibri run a model larger than RAM?
Colibri keeps dense state resident and streams router-selected experts from disk into reusable RAM or VRAM capacity.
What determines whether Colibri feels fast locally?
Local speed depends on expert residency, route locality, storage service, compute, and the bottleneck exposed by the hardware and workload.
Does Colibri’s placement change the model’s computation?
Quality-preserving placement serves the router-selected expert from another tier; route reduction, changed quantization, or different arithmetic needs separate quality evidence.
How should Colibri optimizations be evaluated?
Evaluate them with cold, warm, and changing workloads, end-to-end timing, and the output or quality checks relevant to the change.