Routing Is a Control Problem, Not a Compromise Slogan

For the cloud Opus and private quantized-GLM configurations, routing is the proposed way to trade infrastructure cost against repair risk instead of choosing one backend for every task. The central question is whether a router can make that trade with enough evidence about task stakes, model quality, current capacity, and latency. The answer requires separating an offline cost–quality projection from a live control system that observes what the serving layer is doing.

A hybrid sounds attractive because it seems to offer the best properties of both paths: private capacity for routine work and stronger cloud reasoning for difficult work. But a hybrid is not automatically better than its endpoints. If the local configuration has a higher repair rate for every relevant task tier, sending more work to it buys savings by accepting more correction. And if the router ignores queue state, a theoretically suitable model may still miss the request’s latency target.

The static hybrid is a frontier, not a free lunch

The enterprise study tests this idea through an offline replay of the 613 non-merge commits observed across the two periods. It assigns each difficulty tier to one backend and uses the measured per-tier repair rates to estimate both repair labor and infrastructure cost. This creates a common workload for comparing policies, rather than pretending that the local period’s 367 commits and the cloud period’s 246 commits were identical streams.

The pure API policy has the best defect profile in the replay, with an estimated FCR of 49.1%, but the highest true TCO at $11,419. Pure local GLM lowers estimated TCO to $7,388 while raising FCR to 73.4%. A policy that sends only high-difficulty work to Claude sits between them: $8,412 and 64.6% FCR, a 26.3% saving relative to pure API. Sending high- and medium-difficulty work to Claude costs $9,890 and produces 57.9% FCR, a 13.4% saving.

The ordering expresses a trade-off rather than a hidden optimum. In the measured data, GLM’s fix rate exceeds Opus’s within every tier. Moving a tier from the API to the local backend therefore reduces the infrastructure component and increases expected repair labor and overall repair share. The high-to-Claude policy can be defensible because it protects the work the organization considers most consequential while retaining some private-capacity savings. It does not dominate pure API on quality, and it does not dominate pure local on cost.

That distinction matters for the meaning of “hybrid.” A router can be cheaper because it sends work to the backend with the lower allocated compute cost. It can be safer for selected tasks because it reserves the stronger backend for them. But it cannot claim to improve both cost and quality unless the routing rule finds tasks on which the local backend is at least as effective, or unless escalation and validation add a benefit not present in the static replay. The measured tier rates do not show that condition. They show a monotonic cost–quality frontier.

The replay’s numbers are projections, not invoices from a running gateway. To make all four policies comparable, it divides each backend’s measured cost by the pooled commit count and assumes that infrastructure scales linearly with assigned work. It also treats a backend’s observed repair rate within a tier as transportable to tasks that would have gone to the other backend. A high-difficulty task originally handled locally is assumed to inherit the observed API high-tier rate if the policy reroutes it. That may be a useful first approximation, but task content and model interaction are not interchangeable merely because their change footprints share a label.