NOIR’s Boundary: Local Encoder, Cloud Middle, Local Decoder
This chapter examines NOIR as a system rather than as a collection of privacy features. The specific question is what happens when an open-source LLM is divided between a client and a cloud: which computation and state stay local, what crosses the interface, and why does that division protect prompts and generated code only under particular assumptions? Tracing those transitions gives the reader a way to judge the architecture by its observable surfaces, not by the novelty of its names.
A three-part model of cloud-assisted generation
NOIR starts from an open-source language model and divides it into three consecutive parts. The client holds a lightweight encoder made from the first few attention blocks. The cloud hosts the middle blocks, which contain most of the model’s computation and latent knowledge. The client holds a decoder made from the last few attention blocks. The division is useful only if the three parts still compose into a functioning model: the encoder must produce an input the middle can use, and the decoder must turn the middle’s output into code.
The inference path begins with a prompt on the client. The client’s tokenizer turns the prompt into tokens and looks up their embeddings. NOIR applies its privacy-preserving vocabulary on the client side before the prompt enters the local encoder. The encoder transforms the sequence into a representation that is sent to the cloud. The cloud processes that representation through the middle blocks and returns an enriched embedding. The local decoder then continues the computation and produces the code on the client.
The cloud therefore contributes the expensive central part of the model without receiving the original prompt as readable text. The client does not ask the cloud to return a completed source file. It receives an intermediate result and performs the final decoding step itself. This is the architectural reason generated code can remain on the client even though the cloud supplies most of the model’s capacity.
The split also explains why NOIR relies on open-source LLMs. A client must know enough about the model’s vocabulary, embedding interface, and attention-block boundaries to hold and operate the local pieces. The source argues that open-source models make this feasible: providers can expose compatible components while keeping a service based on closed models separate, and lightweight local blocks require less hardware than hosting the entire model. The proposal is not to copy a proprietary model onto every client. It is to divide a model whose components can legally and technically be distributed.
The architecture creates two different notions of ownership. The client owns the prompt, private training examples, locally held vocabulary and tokenizer, and the code produced during inference. The cloud owns or operates the middle computation. During fine-tuning, the client can update its encoder and decoder locally. The cloud may keep its middle blocks fixed or update a lightweight LoRA adaptation. The important point is not that every parameter is in one place; it is that the local state needed to interpret the client’s data and produce its output need not be disclosed to the cloud.
A useful way to read the system is to follow the object at each transition:
- Prompt text and code begin as client-held data.
- Token identities and embeddings are prepared by client-held vocabulary and tokenizer state.
- An encoded representation crosses to the cloud for middle-block computation.
- An enriched representation crosses back to the client.
- Decoder states and output token interpretation remain local while code is generated.
This sequence is not a proof of privacy. It is an interface map. Every object that crosses the interface, and every secret needed to interpret it, becomes part of the security analysis.
The diagram’s most important edge is not the arrow from the cloud back to the client. It is the representation that travels into the cloud. That representation must be rich enough for the middle blocks to add useful context, yet difficult enough to interpret that it does not become a substitute for the original prompt. The local decoder matters for the same reason at the other end: if the cloud receives the final token sequence or can map output indices through a shared vocabulary, it may recover code even though the prompt was protected.
Local encoding and local decoding also have different failure modes. If the encoder’s output distribution no longer matches what the cloud’s middle blocks expect, the cloud may enrich a representation that is malformed from the model’s point of view. If the enriched representation does not match the local decoder’s input distribution, the client may receive a technically private but unusable result. These are not merely quality bugs. In a split model, privacy transformations alter the interfaces that make the model function, so utility and security are coupled to the same boundary.
What crosses the interface, and what assumptions hold it together
During inference, the cloud sees an encoded prompt representation and performs the middle computation. It can see its own model-side state and the messages required by the service. It does not receive the client’s raw prompt or the code produced by the local decoder, according to NOIR’s design. The distinction is precise but limited: the cloud does not see those objects in their intended form; it still sees an intermediate object designed to preserve information useful for generation.
During fine-tuning, the boundary is more active. The client has paired examples of prompts and desired code. It uses the privacy-preserving vocabulary to encode the prompt, sends the resulting representation through the cloud’s middle blocks, and computes a loss against the desired output. The decoder’s gradient is computed on the client and the backward path reaches the cloud. The cloud can continue the backward computation for its optional LoRA, while the client receives the signal needed to update its encoder. The raw prompt and code remain with the client, but gradients become part of the interface state.
Consider one private example rather than the whole training set. The client prepares an encoded request for a coding task and the cloud transforms it through the middle blocks. The local decoder produces the logits from which the client can evaluate the desired next token or output sequence. That error signal travels backward far enough for the cloud to update its optional adaptation and far enough for the client to update its own encoder. The system does not need to transmit the example itself, but it does need to transmit the consequences of the example for a shared computation. Those consequences are precisely why the gradient path belongs in the threat model.
The split also creates interface contracts that are easy to miss in an architectural sketch. The encoder’s output must have the shape and distribution expected by the first cloud block. The cloud’s enriched output must be a valid input for the local decoder. The tokenizer must assign indices consistently with the embeddings used by the encoder, and the decoder’s output indices must be interpreted consistently by the client. A privacy transformation can preserve the number of dimensions while changing the statistical relationships inside them. From a systems perspective, that is an interface change even if no network API has changed.
This is why the local tokenizer and local model pieces cannot be treated as ordinary configuration. A tokenizer maps indices to token identities. If the cloud knows the same mapping used by the client, an output index can be interpreted as a word or code fragment. A local encoder and decoder contain the transformations that connect the private data to the cloud-visible representation and back to the generated result. The source’s privacy claim depends on the client keeping these pieces, along with the privacy-preserving vocabulary, out of the cloud’s reach.
The cloud may also choose not to fine-tune its middle-block LoRA. That lowers the cloud’s training burden and, according to the paper, can produce only a marginal utility loss in the evaluated setting. But leaving the middle fixed does not make the representation automatically safe. It changes the cloud’s adaptation capability. A fixed middle may reduce one avenue for modeling the privacy transformation, while a trainable LoRA may improve task performance but give a curious provider more opportunity to study patterns across examples. The system’s protection and utility are thus affected by a decision that is partly operational and partly adversarial.
NOIR states an honest-but-curious threat model. The cloud follows the specified protocol but tries to reconstruct prompts, training data, or generated code from what it can observe. This model rules out some behaviors and includes others. It includes analysis of embeddings, output-side information, and back-propagated gradients. It does not assume that the cloud will crash the service or arbitrarily rewrite the protocol. The distinction matters because a system designed for a curious participant is not automatically secure against a malicious participant who can choose inputs, alter updates, or deliberately probe the client.
The source also excludes compromised employees in the client organization. An employee who can disclose the raw prompt, generated code, tokenizer, privacy-preserving vocabulary, or local model parameters can hand the cloud the very information the split was meant to withhold. That is not a minor caveat about implementation discipline; it defines the boundary of the claim. NOIR protects data from the specified cloud-side observer, not from every party that might gain access to client-held state.
Auxiliary information is another boundary condition. The cloud may have public code, common instruction templates, and observations from many requests. Repeated messages can create patterns even when each individual representation is difficult to invert. The system’s design relies on the client-side privacy transformation and secret token mapping to prevent those patterns from becoming a reliable translation back to ground-truth tokens. If the cloud acquires an oracle that lets it query the client encoder, or if it learns the mapping between private indices and tokens, the architecture no longer has the same meaning.
This does not make the architecture fragile in the ordinary sense. It makes its security legible. A sound system description should say which state is persistent, who owns it, and what triggers a transition. In NOIR, a prompt triggers local tokenization and encoding; an encoded representation triggers cloud computation; the returned enriched representation triggers local decoding; and a training loss triggers a gradient exchange. At each transition, the reader can ask whether the object is raw, transformed, secret, or observable.
The architecture also clarifies what it does not attempt to solve by placement alone. It does not prove that an enriched embedding is semantically empty. It does not prevent the cloud from using its own decoder or model-side knowledge to attack the representation. It does not make a shared open-source vocabulary safe if the client’s randomized mapping is disclosed. And it does not guarantee useful generation if privacy transformations create a mismatch that the training process cannot repair.
That is the correct conclusion from the system map. NOIR changes the location of the most sensitive endpoints: prompt preparation and code decoding are local, while the cloud sees a middle representation. The split reduces direct exposure, but the interface remains an object of inference and the secret local state remains part of the protection. Before asking whether the privacy mechanisms work, it is possible—and necessary—to audit this boundary on its own: what is protected by placement, what remains visible, and which assumptions turn a promising architecture into a qualified privacy claim?