A Model Can Fit and Still Be the Wrong Fit
This chapter focuses on the decision that llmfit makes after estimating memory: which execution path should carry the model, and how much headroom does that path have? The goal is to read a run mode and a fit level together. A model that fits on a CPU, a model that fits on a GPU with room to spare, and a model that barely loads are all different practical outcomes even when they share the same model identity.
The distinction matters because “fit” is not a synonym for “fast.” llmfit treats memory compatibility as one dimension and speed as another. First it asks which resource pool can hold the selected configuration. Then it grades how full that pool is and caps the result when the execution path cannot deliver what the strongest label implies. This keeps a comfortable fallback from being mistaken for an optimal path, while still acknowledging that a model that runs in system RAM is more useful than one that cannot run at all.
The path determines which memory matters
The preferred path is full GPU residency. When the model configuration fits in the available VRAM, llmfit treats the GPU as the pool being graded. That path keeps the weights and runtime state close to the accelerator and is normally the strongest performance option represented by the fit model. On a system with several usable GPUs, the analysis may consider their aggregate VRAM when model splitting is supported, but that capacity assumption still depends on the runtime’s ability to divide the work.
A discrete GPU creates a clear boundary between VRAM and system RAM. If the model does not fit in VRAM but a supported quantized configuration fits in system RAM, llmfit can choose a CPU-offload path. Part of the model remains associated with the GPU, while the rest crosses into system memory. The model may be runnable, but the boundary changes the cost of accessing its data. The path is therefore not equivalent to a smaller fully GPU-resident model with the same nominal memory requirement.
Mixture-of-Experts models add a more specialized path. When a complete model is too large for VRAM, llmfit can place active experts in VRAM and inactive experts in system RAM, provided both pools can accommodate their respective portions. The offloaded path is possible because only a subset of experts is active for a given token, but it introduces a dependence on system-memory bandwidth and runtime behavior. If the full quantized MoE model fits in VRAM, the tool prefers the GPU path instead, avoiding that offload bottleneck.
If no usable GPU is available, llmfit falls back to CPU-only execution when system RAM can hold the configuration. The model is loaded into system memory and the CPU performs the inference. This is a real execution path, not the same thing as a failure, but it has a different speed expectation. If the GPU is detected but its VRAM cannot be sized, the analysis also becomes more conservative rather than inventing a precise GPU capacity. An unknown pool cannot support a confident fit claim.
Unified memory changes the path model again. On Apple Silicon and other recognized unified-memory systems, CPU and GPU share one pool. There is no separate system-RAM reserve that can serve as a spill destination in the way it can for a discrete GPU.