Why ICAP Is More Than a Peripheral
For the Silicon Heist study, the key mechanism is not simply that an FPGA can be reprogrammed. The question is who can initiate reconfiguration while the device is already running, and what that path can reach. This chapter follows the AMD UltraScale+ configuration lifecycle far enough to show why ICAP—the Internal Configuration Access Port—is a control-plane interface inside the running design. The goal is to make the later privilege escalation intuitive: a customer does not begin with direct JTAG or eFuse authority; the first decisive change is gaining a route to the configuration engine.
A bitstream is a set of instructions for hardware
A bitstream is the packaged description that configures an FPGA. It contains data for the configurable fabric, but it also contains commands that tell the device’s internal configuration engine how to process that data. The engine interprets the sequence, selects addresses, writes configuration information, and performs the transitions needed to make the new hardware state active. Thinking of a bitstream as a large, passive memory image misses the fact that it has an execution-like structure: commands control how configuration data is applied.
The data is organized into frames, which are the device’s addressable units of configuration. The Silicon Heist paper describes an AMD UltraScale+ frame as 93 words of 32 bits each. The precise size belongs to that device family; the more important point is the relationship between a frame and the physical fabric. Frames are not arbitrary chunks scattered without meaning. They map to portions of logic, routing, and other configuration state. Writing selected frames can therefore change selected physical regions without rebuilding the entire device.
The normal configuration process starts with an empty or reset device and loads the provider shell and the rest of the approved image. After that initial state is active, AMD UltraScale+ devices support partial reconfiguration. A partial bitstream targets a defined region while the rest of the design continues operating. A provider can use this capability to update an accelerator, replace a function, or make more efficient use of a physical resource without resetting every interface on the chip.
Partial reconfiguration creates a useful separation between two kinds of change. The shell and other static logic are expected to remain stable. A dynamic region can be replaced as long as the new configuration fits the region’s boundaries and preserves the interfaces on which the static logic depends. The cloud service relies on those boundaries to turn a physical device into a sequence of tenant workloads. The provider does not need to revalidate the entire chip for every customer update, but it does need confidence that the update cannot reach or redefine the machinery outside the permitted region.
That confidence depends on the configuration engine. The engine is the component that turns configuration data into a changed hardware state. Under the intended cloud workflow, the provider’s shell uses an approved path to request changes to the customer region. The customer controls the design being loaded, but not the authority that decides how arbitrary configuration operations are interpreted or where they are sent. The distinction resembles a storage service in which a tenant may write to an assigned volume but cannot issue commands to rewrite the storage controller itself.
The analogy becomes more precise if we separate data from authority. A customer bitstream contains the desired behavior of the tenant region. The configuration engine owns the transition from encoded data to physical state. The shell mediates that transition and preserves the interfaces needed by the host and the service. A security check that verifies the data is valuable, but it is not a substitute for restricting who can speak to the engine after the data has been accepted.
The provider’s desired transaction has an authority sequence: an untrusted customer supplies an artifact, a trusted service examines it, and a trusted programming path asks the device to apply it. Partial reconfiguration does not remove that sequence; it makes the sequence repeatable for selected regions. The customer may receive a new accelerator image, but the provider still owns the decision that the image is allowed and the path that applies it. ICAP changes the sequence by moving the ability to request configuration into the running hardware. The customer design is no longer only the object being checked. It can become an actor that initiates a later state transition.
That actor-versus-object distinction explains why frame-level access is security-sensitive even when a frame is small. A frame may encode a local change to logic or routing, but the configuration commands surrounding it determine how the device interprets and applies the change. The study does not need a customer to rewrite the whole device to establish a boundary failure. It is enough that a tenant-controlled path can select and apply configuration information outside the provider’s intended transaction. The resulting state may still be limited to a secondary SLR, yet it was reached through authority that the original submission model assumed only the provider possessed.
Configuration also has lifecycle state. A region may be empty, active, being reconfigured, or waiting for the sequence that returns it to normal operation. The device’s configuration engines are connected across the SLRs described earlier, so an update can depend on where its frames occur and how the chain propagates them. The researchers report that making an additional ICAP usable in a secondary SLR required them to account for the chain being interrupted and for the affected region’s operating state. That technical detail matters conceptually: a configuration action is a state transition through a cooperating control system, not an isolated edit to a file.
A safe partial update therefore has an implicit termination condition. The targeted region must receive the intended configuration, the static interfaces must remain usable, and the configuration engines must return to a state in which ordinary operation can continue. The provider’s shell depends on that condition even when the customer never sees it. A low-level interface that can alter the configuration sequence may affect not only the new logic but also whether later operations complete normally. This hidden lifecycle coupling is one reason the paper treats ICAP as a control-plane resource rather than as a convenient implementation detail.
This is why the word “partial” should not be mistaken for “low authority.” Partial reconfiguration narrows the area being changed, which is an important safety property, but the interface that performs the change may still have access to powerful configuration functions within that area. A small doorway can lead to a large room if the doorway opens onto the room’s control system. The security question is therefore about the interface’s authority, not only the size of the region it ordinarily serves.
ICAP turns a running design into a potential control-plane client
ICAP is the internal interface through which logic inside the FPGA can communicate with the configuration engine. In a normal design, an internal interface might move data between a customer accelerator and the shell. ICAP is different because its output is not merely a value consumed by another circuit. It can represent configuration operations that change the fabric itself. Once a running design has a working ICAP connection, the design is no longer only computing on the configuration it received; it may be able to request additional changes to the device.
That is the first privilege boundary in the Silicon Heist chain. At ordinary access, a customer submits a design and waits for provider checks and programming. At configuration-engine access, the running design acquires a route that can operate after the provider’s normal image-generation process. The paper reports that the evaluated tools did not impose a hard limitation preventing an additional ICAP in a secondary SLR, and that the AWS verification pipeline did not prohibit its instantiation. The important discrepancy is between intended placement and enforceable capability.
AMD guidance cited by the paper says that an ICAPE3 configuration component should remain in the static part of the design, alongside other sensitive configuration components. That guidance makes engineering sense: a static component is easier for a provider to own and protect than one placed in a tenant-swappable region. But a recommendation about where a component should be placed is not automatically a security mechanism. If the toolchain can produce a valid enough configuration and the provider does not check for the component, the hardware may still activate it in a location the service intended to reserve.
This distinction is common in systems engineering. A convention can be effective when every producer follows it and every deployment path enforces it. It becomes fragile when an untrusted participant can submit a design and the convention is the only barrier separating that design from a privileged interface. The study’s result does not require assuming that the vendor guidance was meaningless. It shows that the guidance did not, in the evaluated flow, prevent a determined tenant from reaching the capability it was meant to keep out of the dynamic region.
Once ICAP is available, its consequences are both about reading and writing. The paper describes readback access to configuration and user data in a reachable SLR. Depending on the design, that state can include customer logic, third-party intellectual property, internal flip-flop values, or sensitive data held in the fabric. Write access is more fundamental to the escalation: it can introduce or alter circuitry after the provider has completed the checks that were intended to reject it. The authors used a small ring oscillator as a non-destructive demonstration of injecting circuitry that normal provider checks would ordinarily prohibit.
The ring oscillator is useful because it makes the control change visible without requiring a destructive action. Its activity could be observed through a counter, showing that the running device had acquired behavior not present in the originally approved design. The paper connects this capability to previously studied threats such as power or thermal abuse, fault injection, side-channel sensing, and hardware Trojan insertion. Those threats are not all newly executed against Amazon hardware in the paper. They are examples of what becomes possible when the configuration engine no longer accepts commands only from the provider-controlled path.
Read and write access create different kinds of exposure. Readback threatens confidentiality: configuration state may reveal a customer’s design, the state of logic in a co-located region, or information held in registers. In a multi-tenant arrangement, that could become intellectual-property theft or data leakage, although the paper treats such sharing as hypothetical for major providers. Write access threatens integrity and availability: it can change the behavior of the reachable fabric, undermine a monitoring assumption, or insert circuitry that the provider would have rejected before deployment. Neither form requires the attacker to become an administrator of the host operating system. The authority is in the hardware path.
The case also shows why runtime monitoring alone is a complicated answer. A monitor may notice excessive power or temperature, and the provider may gate the customer clock or shut down the FPGA. But a configuration path can change the source of a clock, alter reachable monitoring logic, or produce behavior that does not cross a simple threshold. The paper does not prove that every monitor can be defeated. It demonstrates the design problem: a detector is only as trustworthy as the low-level state it can observe and the controls it can still command. If those controls share the same configuration boundary as the tenant design, monitoring is not independent of the thing being monitored.
ICAP is also bounded. On the AWS devices examined, the shell’s critical logic was in lower SLRs, while the customer-reachable ICAP was in SLR2. The directional configuration chain did not let the researchers move backward to every earlier region. The result was not instant ownership of the entire FPGA. It was a new control relationship over a reachable secondary region, with the possibility of downstream effects. A precise security analysis must retain both facts: reachability was limited, and the reachable interface was far more powerful than an ordinary tenant datapath.
The configuration lifecycle in the diagram has two paths. The first is the provider’s expected path: a customer design becomes an approved image, the image supplies commands and frame data, and the configuration engine applies them to a selected region. The second begins after the design is running. If the design contains a usable ICAP, it can become a client of the same engine. The provider’s original verification step still exists, but it is no longer the only source of configuration requests.
That temporal difference is the heart of the mechanism. A submitted design can pass checks because it contains no forbidden behavior in the form the provider inspects. A later ICAP transaction can change the physical state without resubmitting the resulting configuration for the same checks. The control plane has moved from “the service receives and verifies a customer artifact” to “the running customer artifact can ask the device to modify itself.” The boundary has not necessarily disappeared everywhere, but it has shifted to an interface that the original model did not adequately govern.
The next privilege transition follows naturally from this one. JTAG is another interface into low-level FPGA functions, normally associated with testing and debugging rather than ordinary tenant operation. On the studied devices, internal MASTER_JTAG circuitry exists alongside the configuration resources, and eFuses hold security-relevant non-volatile state. Directly instantiating that circuitry was blocked by a design-rule check. With ICAP control, however, the researchers report that the required configuration changes could be activated after deployment. The question is no longer whether a tenant can alter a dynamic region; it is whether that altered region can reach the device’s security and recovery controls.