What Colibri Proves—and What It Still Has to Measure
JustVugg/colibri should be judged against the problem it actually addresses: whether a frontier-scale sparse model can be made executable by placing its weights across storage, RAM, and VRAM without silently changing the model. This final synthesis connects the performance evidence, operator workflow, application interfaces, family architecture, and unresolved experiments. It gives readers a conditional answer about when Colibri is worth trying, how to test it responsibly, and why the project demonstrates a viable placement approach rather than universal fast local frontier inference.
When the hierarchy pays
The strongest supported claim is about feasibility under sparsity. Colibri’s reference GLM-5.2 path keeps dense and shared state resident, treats the routed experts as a large disk-backed set, and moves selected copies into RAM or VRAM when capacity and locality justify it. That architecture removes an all-or-nothing requirement: the complete model does not have to fit in the fastest tier at the same time. It does not remove the need for a large disk, a valid checkpoint, sufficient runtime memory, or patience during cold execution.
Whether the result is useful depends on the relationship among three quantities: how much of the routed set can be resident, how often the workload selects the resident part, and how quickly a cold expert can be served. A 25-GB development machine with a roughly 1-GB/s drive is a valid demonstration of the capacity idea, but the documented cold rate is only about 0.05–0.1 tokens per second. A machine with enough RAM to keep most experts warm can move the bottleneck away from storage and toward CPU computation or RAM bandwidth. A large heterogeneous host can go further, but only when its VRAM and interconnect actually reduce the work on the critical path.
The benchmark record supports this conditional reading. On one machine, swapping a 1.51-GB/s QLC drive for an 8.81-GB/s PCIe5 drive raised throughput from 0.10 to 0.28 tokens per second and changed the profile from mostly disk time to mostly matmul time. On a host with roughly 430 GB of RAM and a 98% expert hit rate, disk was effectively removed and compute became the limiting resource. On the six-RTX-5090 experiment, full residency across VRAM and RAM removed disk waits and produced 5.8–6.8 tokens per second in the specified greedy runs; the remaining profile was dominated by expert computation on the CPU. These are not contradictory reports. They are observations of the same hierarchy at different bottlenecks.
The first decision rule is therefore simple: use Colibri when the value of running the full sparse model locally outweighs the cost of its cold storage path and setup. If the model cannot be stored safely or the machine cannot reserve dense state, context state, and working memory, the placement idea has no foundation. If it can run but most routed experts remain cold, expect a capacity-enabled experiment rather than hosted-service responsiveness. If repeated or related workloads create a high hit rate, the learning cache and extra RAM may make the system substantially more practical.
A second rule concerns what to improve first. When profiles show foreground disk wait, measure storage, cache residency, and the shape of expert reads before optimizing matrix arithmetic. When the cache is warm and matmul dominates, test CPU kernels, physical-core scheduling, RAM bandwidth, or a suitable accelerator. When a GPU is present, confirm that its memory is qualified for placement and that the measured profile shows it reducing the binding work. Hardware prestige is not a bottleneck diagnosis.
The evidence also says that capacity should be added at the right granularity. A second drive can help substantially when the engine uses it to parallelize the large expert read that blocks a layer. Merely splitting whole shards may offer much less because the existing request concurrency has already exposed the available parallelism.