How AEGIS Turns Training Redundancy into Evidence

AEGIS does not need to create all of its checking work from scratch. Large-model training already repeats some computations to save memory or make kernels efficient. This chapter examines when that existing repetition is strong enough to support a bitwise comparison, how AEGIS captures the comparison as a compact fingerprint, and why it instruments only selected points in a recomputation chain. The result is a different kind of evidence from an approximate checksum: exact equality is powerful here, but only when the two executions really are self-equivalent.

Redundancy already exists in the training computation

A computation is self-equivalent when the same operation is executed twice with the same inputs and execution configuration and is expected to produce the same output. Under deterministic execution, the outputs should be bitwise identical. That gives AEGIS a direct test: record a compact representation of the first result, record the corresponding representation of the second, and treat a mismatch as evidence that the two executions did not agree.

The requirement is stronger than “the two values are close.” In the algorithmic checks described earlier, a checksum difference is interpreted against a tolerance because low-precision arithmetic naturally creates numerical variation. A self-equivalence check relies on a different property. If the computation is deterministic, equality is expected exactly at the bit level. A mismatch is therefore more decisive for that protected segment, while a comparison between executions that are not genuinely equivalent would be difficult to interpret.

AEGIS uses an xorsum reduction to turn a high-dimensional output into a compact fingerprint. The reduction is not a second copy of the tensor and is not intended to preserve every detail of the output for later inspection. It is a small signature that can be recorded during one execution and compared with the signature from its equivalent execution. This keeps the inline work and retained state smaller than a full tensor comparison.

FlashAttention creates a comparison pair

FlashAttention provides an operator-level opportunity. To reduce memory use, its backward pass recomputes intermediates that were produced during the forward pass instead of retaining all of the large intermediate matrices. The forward execution produces an output or intermediate, and the backward execution later recomputes the corresponding value when the necessary input blocks are loaded. That repetition was introduced for training efficiency, but it also creates a natural correctness comparison.

AEGIS can apply an xorsum reduction to the relevant forward output and save the first fingerprint. When backward recomputes the corresponding segment, it applies the same reduction to the recomputed output. cVerifier later compares the two fingerprints. If they differ, the two executions that should have agreed did not. The detector has used work the training job already needed, rather than launching a second complete training task merely to test the GPU.

The distinction between “recomputed” and “duplicated” matters. Activation recomputation is already part of the memory/compute trade-off in some training frameworks: the framework releases selected activations and calculates them again when backward needs them. AEGIS adds a compact fingerprint around that existing relationship. It does not claim that all training redundancy is deterministic or that every repeated operation automatically becomes a valid check.