The Race Before the Log Leaves the Kernel

HyperAudit’s problem diagnosis starts with a failure that is easy to miss: a system call can be recorded and still not be safe evidence. The specific question here is where the race window comes from in a kernel-local auditing path, how a privileged attacker turns that window into missing history, and why the conventional fix makes the workload pay for protection on every event. Following that chain gives us a concrete requirement for any architecture that claims to preserve audit evidence after a guest kernel is compromised.

Three times in one audit event

A system call is a request from a program to the operating system—for example, to open a file, create a process, or send data. The kernel receives that request, performs checks and work on the program’s behalf, and returns a result. Because the kernel sees the request and often has the surrounding context, it is a convenient place to observe it.

But “observe” is not one instantaneous action. For an audit event, at least three times matter. Event time is when the operation occurs and the kernel has the facts that describe it. Collection time is when an auditing component copies or assembles those facts into a record. Commit time is when that record has crossed into a destination the attacker in the guest can no longer alter. Between collection and commit, the record is usually in a buffer. The buffer is not an implementation footnote; it is a temporary owner of the evidence.

That distinction explains how an event can be true but not trustworthy. If the collector has placed a record in memory that remains accessible through the kernel, the record is still provisional from a security perspective. It may be complete enough to look convincing, yet remain exposed to a later change in the system’s privilege state. A remote database can preserve the bytes after they arrive, but it cannot retroactively protect bytes that never arrive or that were altered before submission.

A useful diagnosis therefore starts by asking four questions of the path: who creates the event, who assembles the record, who can write the buffer, and what action makes the record durable? If one component controls all four—and that component is the very kernel that may be compromised—the path has hidden coupling between execution and evidence protection.

This distinction also separates the root cause from nearby symptoms. A slow disk can cause backlog, and a poorly chosen event policy can leave an analyst without useful context, but neither explains why a privileged guest attacker can selectively rewrite records that are already waiting to be submitted. Likewise, sending data over a protected channel may secure the transport while leaving the source buffer mutable. HyperAudit’s diagnosis is narrower and more structural: the danger follows from allowing the component that runs the compromised environment to retain authority over the collector and its pending state. Storage, policy, and transport still matter, but they operate after or around that first boundary.

The race condition is a race over authority

HyperAudit’s authors describe a race-condition attack against auditing frameworks whose collectors are visible to or dependent on the guest kernel. The attacker first gains kernel-level privilege through a vulnerability or another compromise path. Once there, the attacker does not need to forge an entire remote archive. They can target the records that are still waiting in the host or kernel buffer and remove events connected to the compromise before user-space processing or secure submission completes.

The important point is temporal, not merely hierarchical. A pre-compromise event may already have happened. The auditing system may even have started to process it. Yet if its record remains in guest-controlled memory, the later kernel compromise gives the attacker an opportunity to change the historical account. The attacker wins by arriving before commit, not by proving that the original event was false.

The paper characterises the relevant exposure in existing systems as typically lasting from about 10 to 250 milliseconds, while prior defences try to reduce the time in the buffer—often to less than 5 milliseconds—through per-system-call or periodic submissions. Those figures describe the systems and conditions discussed by the authors; they are not a universal timing rule. The general mechanism does not depend on a particular number. Any nonzero period during which an attacker can both obtain the necessary privilege and reach the uncommitted record is a possible evidence boundary.

The attack also illustrates why the word “pre-compromise” matters. Suppose a process opens a sensitive file, creates a new child, and then an attacker takes control of the kernel. If the records for those earlier actions are still in a mutable buffer, removing just the creation or permission-change event may make the later activity appear to have no origin. The attacker need not erase everything. Selective deletion can break the chain an investigator relies on to connect cause and consequence.

This is why a healthy-looking remote stream cannot prove completeness by itself. A collector may continue forwarding plausible records after the compromise while quietly losing the events that would reveal what happened. The absence of an entry is difficult to distinguish from the absence of the underlying action unless the collection path can account for its own loss and protect its pending state.

The attacker’s advantage comes from control over the recording machinery, not from unusual insight into the application. The kernel already mediates the relevant operations and controls the memory in which many collectors keep their working state. Once the attacker has the same level of privilege, the collector cannot reliably distinguish a legitimate kernel update from a malicious one unless some higher authority enforces that distinction. Even a record that contains a correct timestamp and a correct system-call name can be untrustworthy if the attacker can decide whether that record survives the next few instructions.

This also explains why reducing the window is not the same as removing it. A shorter interval lowers the number of pending records exposed at any instant, but it leaves the collector in the attacker’s authority domain. The defence is still racing the compromise. If the attacker can reach the buffer, the success of the defence depends on timing, scheduling, and load; if the boundary itself prevents access, the security argument no longer rests on winning a race for every event.

sequenceDiagram participant App as Guest application participant K as Guest kernel participant C as Kernel-local collector participant B as Uncommitted buffer participant A as Attacker after kernel compromise participant R as Secure receiver App->>K: Issue system call K->>C: Expose event to collector C->>B: Assemble pending record Note over B: Collection complete, commit pending A->>K: Gain kernel privilege A->>B: Inspect, alter, or delete pending records B-->>R: Submit what remains R-->>R: Process and persist

The sequence makes the failure visible: the attack is not necessarily a race against the application’s original action. It is a race against the transition from mutable buffer state to protected evidence. A design that moves only the receiver or storage farther away leaves that transition in the attacker’s reach.

Flooding turns a finite buffer into a lever

The second failure mode is a flood attack. An attacker generates a large number of benign system-call events, not necessarily because those calls are interesting, but because they consume the auditing system’s capacity. Every collector has finite CPU time, memory, buffer space, and downstream I/O. If events arrive faster than the path can process and commit them, backlog accumulates.

At that point, the system has two unattractive choices. It can drop events to keep the workload moving, or it can make the workload wait until the auditor catches up. Dropping events damages integrity directly. It creates gaps that may include the attacker’s meaningful actions, and it makes the later archive ambiguous: silence could mean that nothing happened, or that the collector could not keep up. Blocking preserves more of the record, but it turns the audit mechanism into a source of latency and reduced throughput.

The danger is not limited to a dramatic denial of service. A modest, sustained increase in event volume can change which records survive. A high-value event may be surrounded by enough routine calls that it is evicted, delayed, or processed after an operational deadline. If ingestion is delayed, detection is delayed too; a defender may receive the evidence only after the opportunity to contain an intrusion has passed. The Australian Cyber Security Centre’s guidance treats timely ingestion and storage capacity as security concerns for exactly this reason.

HyperAudit’s paper reports that existing frameworks often use blocking mechanisms to preserve log integrity under large-scale flooding, but also notes the resulting performance degradation. The flood therefore exposes the same architectural tension as the race condition from a different direction. In the race, the buffer is dangerous because the attacker can change it. In the flood, the buffer is dangerous because it is finite. A design that makes the buffer harder to tamper with still needs a plan for what happens when producers outrun consumers.

The attack can also amplify the race. If a slow submission path leaves more records pending, it increases the amount of evidence that is available to a kernel-level attacker at the moment of compromise. If the system responds by blocking frequently, the attacker gains a performance lever even when no records are lost. Security and availability are not separate checkboxes here; they are competing outcomes of the same queue and the same authority boundary.

The operational consequence is a question of declared loss. A well-designed system may choose to prioritise some events, store others at a lower tier, or apply backpressure, but it must make that choice visible to the people who will interpret the archive. Silent loss is especially damaging because it preserves the appearance of a continuous record. The ACSC guidance therefore connects retention and timely ingestion with detection and incident response: a record that arrives too late, or a gap whose cause is unknown, can be as consequential as a record that was never requested. HyperAudit’s later performance tests matter against this diagnosis because they ask whether moving the collector’s authority can reduce the need for blocking without turning the buffer into an unbounded promise.

Why the conventional fix costs performance

Prior systems respond to the race by shortening the time that records remain exposed. A per-system-call design submits or protects each event as it is produced. A timed design commits at regular intervals. A priority-based design gives selected events earlier treatment. These strategies can be sensible responses when the collector and its buffer must remain in the kernel. They reduce the number of uncommitted records an attacker can reach, and they may prioritise evidence judged more valuable.

They also add work to the path that every application call must traverse. The collector has to perform checks, schedule or trigger a submission, cross a privilege boundary, rotate or allocate buffers, and coordinate with a receiver. HyperAudit’s authors describe these crossings as a significant source of overhead, particularly for syscall-intensive programs. In a system where most calls are ordinary and no compromise is occurring, the workload still pays for the defence intended for the exceptional case.

The cost is not only the time of one submission. Synchronisation can create contention among CPUs and threads, wake sleeping user-space components, and make the application wait for a receiver or storage path. A design that protects integrity by blocking can also make a flood more effective, because benign volume consumes the same scarce coordination capacity needed for high-value events. The system may be secure in the narrow sense that it refuses to drop data, but too slow to be acceptable in production.

Priority-based submission introduces another trade-off. It can give security-sensitive calls earlier treatment, but the policy must know which calls will matter before an unfamiliar attack occurs. A low-priority event may be the missing link in a later investigation, and a malicious actor can generate routine activity to compete with the events that the policy prefers. Timed submission is simpler to reason about, but its safety depends on the interval and the workload’s ability to keep the buffer within that interval. These designs are not irrational; they are compensations for a boundary that remains exposed. Their costs and residual uncertainty should be evaluated together.

This does not prove that asynchronous collection is always better. It identifies the condition under which frequent submission is being used: the collector is exposed because it depends on the kernel and shares its authority. If that condition changed, the system might not need to make every event cross a boundary immediately. The performance argument is therefore downstream of the security diagnosis, not an independent preference for speed.

The requirement left by both failures

Race-condition and flood attacks point to one shared design requirement. The collector and its pending records must be protected from a compromised guest kernel, while the normal path must move data efficiently enough that finite buffers do not become an easy source of loss or blocking. Protecting only the final storage misses the race. Making every event synchronous addresses the race at a recurring cost and may worsen the flood.

That requirement is deliberately narrower than “make logs trustworthy.” It says that an auditing architecture must specify who controls each evidence state and what happens at capacity. It must also say which period its integrity claim covers. The paper’s threat model focuses on preserving logs generated before the compromise time; it does not provide the same forensic guarantee for events generated after a kernel attacker has taken control.

HyperAudit’s proposed answer is to move the collection authority beyond the guest kernel and use a protected asynchronous path. The next question is not whether that sentence sounds secure, but how it can be made real: how can a hypervisor inject a collector the guest cannot freely inspect, enforce permissions on its code and buffer, and still let the collector see enough of each system call to produce useful evidence? Those are the mechanisms that determine whether the new boundary is more than a change of diagram.

References