Overview
Magnitude makes local agent inference a managed decision loop: discover candidates, bind an exact artifact to a serving profile, test fit against the machine, and rank viable trade-offs before download. The daemon preserves service, model, and session state; the serving path explains queueing, prefill, decode, streaming, and conditional acceleration; integrations determine how much control and progress a harness can expose. Its value is hardware-aware local control and a bounded privacy path, not a universal performance, quality, or UI guarantee.
Learning outcomes
- Magnitude ranks only configurations that pass required capability and machine-specific fit gates.
- Runtime memory depends on the exact artifact and serving profile, not download size or model names alone.
- ACN, ICN, and the SDK separate service readiness, model residency, and client recovery.
- Progress, benchmarks, privacy, and harness support remain path- and workload-specific rather than universal guarantees.
Keywords
Preview every chapter · Unlock full text, audio, and video from chapter 4
PlansFAQ
How does Magnitude choose a local model?
Magnitude chooses a local model by filtering for required capabilities and assessed memory and context fit, then ranking viable configurations by workload priorities such as speed, intelligence, fidelity, and context. The user makes the final choice before a catalog download.
What does “fits” mean in Magnitude?
In Magnitude, “fits” means native planning found that an exact artifact and serving profile pass the observed machine’s physical-memory and device constraints. Runtime loading rechecks current availability, so fit is evidence under a snapshot, not a guarantee for every later state.
Why can a Magnitude model be installed but not ready?
A Magnitude model can be installed but not ready because installation stores the artifact while readiness requires the service and native runtime to admit and load it under current resources. ACN can be ready while a model is unloaded, and a request can still queue or prefill after loading.
What does OpenAI compatibility not guarantee in Magnitude?
OpenAI compatibility provides a familiar semantic request boundary, but it does not guarantee model control, raw progress observation, or native UI display. Those capabilities depend on the harness, version, and available integration seams.