The Architecture of Confidential AI: From VM to Inference
The NIST reference architecture turns confidential computing from a promise about hardware into a sequence of decisions around one AI workload. This discussion focuses on how a TEE-capable virtual machine, attestation, a relying party, and key management depend on one another. Tracing those interfaces gives the reader a practical way to see why a workload receives a model key only after its environment has supplied acceptable evidence.
The object being protected is not just a file called “the model.” It is a running relationship among model parameters, inference data, workload code, platform state, and the authority that controls access. The architecture succeeds when those pieces meet inside a coherent trust chain. It fails when one stage is treated as a substitute for another—for example, when launching a confidential VM is mistaken for proving that the intended code is running, or when a signed report is mistaken for permission to release a secret.
Four stages, one dependency chain
NIST divides its reference path into four stages. The division is useful not because every confidential-computing system must use the same labels, but because it makes the dependencies visible. A platform capability creates the possibility of isolation. Evidence makes the platform state assessable. Policy turns evidence into an access decision. Key release enforces that decision at the point where the encrypted model becomes usable.
Stage 0: configure a TEE-capable VM. The workload first needs a virtual machine whose hardware and cloud configuration provide the intended confidential-execution capability. Many cloud providers offer such VMs. In the NIST example, a confidential VM is created with a supported image and size, and the resulting environment exposes the TEE capability to the workload.
This stage establishes a condition, not a conclusion. It says that the workload has been placed on a platform designed to isolate selected memory and execution from the host software around it. It does not yet tell the model owner whether the correct workload code was launched, whether the platform is patched, or whether the environment satisfies the policy for a particular secret. A TEE-capable VM can be necessary without being sufficient.
Stage 1: configure remote attestation. The workload needs a way to present evidence that it is using the expected hardware and software state. NIST describes a hardware-backed report containing measurements of the VM’s initial state—such as firmware, bootloader, operating system, and application code—and a platform identifier. The evidence is signed by an attestation key rooted in hardware and sent to a remote verifier.
This stage changes the question from “Was a confidential VM requested?” to “What environment is actually asking for access?” The verifier checks the signature and compares the measurements with known or expected values. The detailed challenge-and-response mechanics matter for freshness and authenticity, but the architectural point is simpler: the workload must produce a claim that an outside party can appraise rather than asking the cloud console to be trusted as the final authority.
Stage 2: configure the relying party. A relying party uses the attestation result to decide what the workload may access. It might be the application owner or a service acting on that owner’s behalf. The relying party can require a particular measurement of the application code, an approved version, an acceptable revocation status, or an intact data pipeline. These are not properties of the VM label. They are conditions imposed by the party that owns the model or the data.
This stage supplies the missing meaning of “trusted.” A verifier can determine that a report is authentic and that its measurements match reference values. The relying party still has to decide whether those values are sufficient for the requested action. A workload may be genuine but unauthorized for a particular model. It may be running approved code but an outdated or revoked platform. It may have the right platform identity but fail a policy about the integrity of the data it will process.
Stage 3: execute the key-release workflow. Only after the earlier conditions are met does the key-management path release the material needed to decrypt the model or its inputs. In the NIST description, the attestation evidence is evaluated against a key-release policy. If the policy succeeds, the service obtains a decryption key from the key-management system, protects the response for the requesting workload, and sends it back. The workload unwraps the response and uses the recovered key inside the TEE to decrypt the model.
The ordering is the security mechanism. If key release happened before attestation, the model could be decrypted by any requester that reached the service. If attestation produced evidence but no relying party applied a policy, the evidence would not control access. If a key were released to a process outside the intended protected environment, model encryption would only delay exposure. The stages therefore form a dependency chain, not a four-item feature list.
Each stage also has a different failure meaning. A failure at Stage 0 can mean that the chosen VM or platform does not provide the required confidential-execution capability. A failure at Stage 1 can mean that the workload cannot produce acceptable evidence of its state. A failure at Stage 2 can mean that the evidence is authentic but does not match the model owner’s policy—for example, the code or platform is not the approved one. A failure at Stage 3 means the key authority has withheld the material needed to make the model usable. Treating all four outcomes as the same “cloud error” would hide where the trust chain stopped and who owns the remedy.
The stages also divide state ownership. The cloud platform supplies the virtual machine and the hardware conditions, but it does not decide whether a particular model key should be released. The workload supplies the request and the evidence associated with that request, but it cannot declare itself authorized simply by making a claim. The verifier assesses the evidence, while the relying party defines the acceptable state and the action that state permits. The key-management path holds the secret and enforces the final gate. This division is what keeps a platform capability from silently becoming an authorization decision.
The sequence is not a promise that every deployment will use four separate services. A single organization may operate more than one role, or a different implementation may package verification and policy together. What must remain visible is the causal relationship: some mechanism must isolate the workload, some trusted process must evaluate evidence, some authority must decide whether that evidence is sufficient, and some key-control mechanism must enforce the decision. NIST’s four-stage structure is valuable because it names those responsibilities without requiring them to have identical organizational boundaries everywhere.
The model owner’s policy is the hidden connector among the stages. It has to refer to a state that the attestation system can actually measure and a condition that the key service can actually enforce. If the owner wants to trust a particular application version, but the evidence cannot distinguish that version, the policy is aspirational rather than operational. If the owner’s reference values are too broad, a report may pass while describing more code or a different configuration than intended. If they are too narrow, legitimate maintenance can cause denial even after a safe update. The architecture does not remove this governance problem; it makes the problem explicit at the point of key release.
The main handoffs look like this:
The sequence shows what crosses each interface. The workload sends evidence to an attestation service, then sends the resulting token with a key request. The relying party or key broker sends a request to the key manager only after its own policy permits that transition. The key manager returns key material to the broker, not an unqualified public copy to the cloud runtime. The broker returns a protected response that the workload can recover with the key material it holds. The final transition—decryption and inference—takes place in the workload’s protected execution path.
One encrypted model, several owners of state
The abstract stages become easier to understand when applied to the NIST prototype. The example uses Intel TDX as the TEE technology exposed through a Microsoft Azure confidential VM. Intel Trust Authority evaluates the TDX quote and returns an attestation token. Intel Key Broker Service, or KBS, acts as the relying-party path for the key request, while a key management system, or KMS, retains the decryption key and applies its own release policy. These names identify the particular prototype; the transferable design is the division of responsibilities.
The workload owns the request for inference and, in the described flow, creates an RSA key pair. It retains the private key. The public key is placed in the attestation-held data so that the key broker can use it to protect the response. This creates a useful binding between the workload that presented the evidence and the response that carries the model key. The key broker does not need to send a generally readable secret merely because the caller can reach its endpoint.
The KBS and KMS also own different decisions. The KBS compares values in the attestation token with the reference values in its key-release policy. If those comparisons succeed, it contacts the KMS. The KMS applies a separate policy of its own before returning the requested key or secret. The separation means that an acceptable platform claim and an acceptable key-custody decision are both required in this example. Neither service is simply a passive courier.
The response is protected in layers. NIST describes the KBS creating an AES symmetric wrapping key to encrypt the KMS-supplied key in the response body. It can also use asymmetric key encapsulation tied to the public key provided by the attester, including the symmetric wrapping key. The workload uses its retained RSA private key to decrypt the response, extracts the symmetric wrapping key, and uses that key to recover the KMS-supplied decryption key. It can then decrypt the encrypted machine-learning model.
The point of this mechanism is not that RSA or AES is a universal confidential-computing recipe. The point is that key custody and execution identity are joined. The model is encrypted before deployment, and its decryption key is held in wrapped form under a policy. The workload must first establish the acceptable state that the policy recognizes. Only then does the protected response cross back to the workload, where the model can become active.
This arrangement gives each state a clear owner. The cloud platform owns the VM’s physical and virtual resources, but it is not supposed to become the plaintext owner of the model. The verifier owns the work of checking the evidence. The relying party owns the judgment about whether that evidence authorizes access. The KMS owns the protected key material and its own release condition. The workload owns the private key used to recover the response and the inference process that consumes the model. The data or model owner must still define the policy that makes those decisions meaningful.
The boundaries also explain what the architecture does not automatically solve. NIST qualifies its protection claims by reference to a patched and attested platform and to normal operation. The design does not show that every endpoint, output, log, or application interface is protected merely because model decryption occurred inside the TEE. Nor does the specific Azure and Intel path establish how another cloud, processor, accelerator, or key service will divide the same responsibilities. The NIST report presents one possible implementation rather than a universal deployment standard.
A sound architecture review should therefore ask two related questions. First, can the platform produce evidence that identifies the environment and code the model owner intended to trust? Second, does the key-release path enforce a policy that is narrow enough to make that evidence matter? The four stages answer those questions in order. A VM creates the protected setting; attestation describes it; the relying party interprets it; and key management makes the interpretation consequential. With that chain in place, the next issue is the evidence itself: how a verifier establishes that a report is authentic, current, and relevant to the secret being requested.