The Real Product Is a Trustworthy Decision Loop
To judge Magnitude as a whole, return to the opening question: what does it take to make local inference practical for an agent rather than merely possible on a computer? This final chapter connects the machine, the exact model artifact, the native inference runtime, the local daemon, and the harness that presents the result. It turns those relationships into a final judgment: Magnitude is most valuable when it makes each decision visible and reusable, while its limits remain attached to hardware, workload, evidence strength, and host compatibility.
The system's value is the connection between facts
The central relationship is not “Magnitude plus a model.” It is a loop between a workload and a machine. The workload supplies requirements: perhaps vision, tools, structured output, reasoning, quick iteration, or a large working context. The machine supplies constraints: memory domains, available devices, placement possibilities, and current free capacity. Magnitude's job is to connect those two inputs to an exact artifact and a serving profile, then carry the choice all the way to an agent request.
The curated catalog begins the loop by giving a configuration a stable identity and useful context. It can distinguish model families, variants, architecture class, context, capabilities, fidelity, and any reviewed draft material for speculative decoding. Immutable artifact references keep the choice reproducible. But the catalog does not know the reader's current memory state or device topology well enough to decide by name alone.
ICN supplies the next link. It inspects the exact artifact, plans the selected serving profile, maps its memory demands onto the machine's physical domains, and produces an assessment. A fitting result is evidence that the plan passed under the observed conditions. A non-fitting or incompatible result is a reason to change the profile or configuration, not a prompt to turn the preference dial harder. The native planner's role is to make resource claims concrete before full acquisition and to recheck the decision at runtime load.
The recommendation layer then turns those facts into a choice a person can make. It filters out configurations that do not fit the machine and exposes the remaining trade-offs among memory, context, expected speed, broad intelligence, artifact fidelity, acceleration, and capabilities. The user supplies the priorities that hardware cannot infer. This is the first place where Magnitude's control-plane idea becomes visible: it does not claim to know which objective matters most; it makes the feasible objective space legible.
After the user chooses, acquisition creates a durable local artifact. Loading creates a resident runtime allocation. ACN owns the product-facing state and the client boundary; ICN owns the native model and inference state. The daemon keeps those facts connected without replacing ICN with a second inventory or asking every client to supervise a native process. A model can be installed but unloaded, a service can be ready without a resident model, and an agent session can be idle while both are healthy. The loop remains trustworthy because each state has a distinct owner and meaning.
The live request completes the connection. The scheduler puts decode work before new prompt work, then shares remaining prefill in rotating quanta. The stream can report loading, queueing, prefill, and generation separately from ordinary message and tool output. A reviewed speculative method can reduce target-model decode work when its proposals are accepted, while the target remains the authority over the resulting tokens. These mechanisms change how a viable configuration behaves; they do not erase the fit decision or transform a draft into a new quality identity.
The harness is the last link, and it is where many “local model” claims become either useful or overstated. A local agent needs more than a socket. It needs a control path for choosing or stopping a model, a semantic path for messages and tools, and—if the user is to see loading or prefill—a supported observation and UI path.