When “Nothing Went Wrong” Is Still Evidence

OS-Sanitizer is a research prototype that uses eBPF to infer latent defects in Linux applications from behavior observed while those applications are running. The question at the center of this series is deceptively simple: can an ordinary event reveal that software could do something harmful, even when no crash, wrong result, or other failure has appeared? The practical goal is to learn how to read that kind of signal—what it tells us, what it only suggests, and what must still be checked by a human investigator.

That question changes the usual starting point for software testing. We are accustomed to asking whether a program failed under a test. OS-Sanitizer asks a prior question: did the program’s behavior, together with its surroundings, expose a condition that may fail under a different set of circumstances?

The problem begins before failure

A conventional dynamic test runs a program and looks for an outcome that violates an expectation. If a test causes a program to crash or produce an invalid result, the evidence is concrete: something went wrong in that tested situation. This approach is indispensable. It gives developers a counterexample they can reproduce and fix.

But it is also naturally limited by the situations the tests exercise. A defect may remain dormant because the relevant ownership, privilege, timing, input, or deployment arrangement never appears in the test environment. The program can behave normally while still carrying a weakness that another user, administrator, dependency, or operating context will expose later.

Static analysis approaches the problem from the other direction. It examines source code and looks for patterns associated with mistakes or poor design, often without needing a failing execution. That broader view is useful, but source alone may not reveal how a particular call is used in the live system. A library routine can look merely questionable in isolation and become dangerous when a privileged application invokes it on data controlled by someone else.

Dynamic defect inference occupies the space between these approaches. It uses runtime evidence, but it does not wait for a failure. Its subject is not an imagined vulnerability detached from an execution. It is a real behavior observed in a real context, interpreted as evidence that a latent defect or code smell may be present.

OS-Sanitizer is more than an eBPF monitor

It is important to keep the subject clear. eBPF is the enabling Linux subsystem: it provides a way to observe and process events from both user space and kernel space. OS-Sanitizer is the analysis built on that vantage point. It adds heuristics that ask whether a combination of events resembles a known defect pattern and a reporting workflow that presents the suspicion for investigation.

That distinction matters because eBPF’s ability to see an event is not the same as OS-Sanitizer’s ability to interpret it. The platform supplies runtime visibility and in-kernel processing. The prototype supplies the questions: which events belong together, which surrounding facts matter, and what combination is suspicious enough to report. A system-wide view is valuable because application behavior does not stop at an application boundary. A user-space request can cause a kernel operation; a library decision can acquire meaning from file ownership; a helper can become risky when a downstream program calls it with elevated privileges.

The central theme is therefore not “more monitoring.” It is the conversion of benign behavior into contextual evidence. The same event may be harmless in one setting and worth urgent review in another. OS-Sanitizer tries to preserve enough of the setting to make that difference visible.

flowchart LR A["Benign Linux runtime event"] --> B["Context: ownership, privilege, and use"] B --> C["OS-Sanitizer heuristic"] C --> D["Prioritized report"] D --> E["Source review or reproduction"] E --> F["Remediation or improved testing"]

The arrows in this model describe an investigation path, not a guarantee. A runtime event does not automatically become a defect. The context makes a heuristic possible; the report focuses attention; source review, reproduction, or additional testing determine whether the suspicion holds.

One operation, several meanings

The paper’s motivating example is a recursive directory-removal routine. At a high level, the routine checks whether a path component is a directory and then opens it so that traversal can continue. Considered only as a short piece of application logic, that behavior may look ordinary. Its security meaning changes when another user can modify an intermediate part of the path, or when a privileged process applies the routine to content owned outside that privilege boundary.

This is the kind of issue often described as a time-of-check/time-of-use, or TOCTOU, problem: the world can change between the check and the later use. Yet even that label is not the whole judgment. If a privileged user owns the entire tree and no one can modify it concurrently, the same implementation may never encounter the dangerous condition in that environment. If a less-privileged user controls part of the path, the operating context can turn an apparently benign routine into a route to an unwanted action.

The distinction is more than a technical footnote. Developers rarely control every context in which a library function or utility will eventually run. A function may be written with one ownership model in mind and later become part of container-management software, a build system, or an administrative tool. Downstream users may rely on a stronger guarantee than the original implementation can provide. A weakness that stays invisible in ordinary testing can become consequential at the boundary between users, processes, and the kernel.

OS-Sanitizer’s path-oriented analysis is designed around that boundary. It does not merely ask whether a path operation occurred. It gathers contextual information about the path and the permissions around it, then reports when the combination indicates that the operation could be intercepted or redirected. That is a more informative question than “did the program call a file API?” It is also a narrower one than “is this program vulnerable?” The report supports the first judgment, not automatically the second.

A defect is not the same as a failure

To understand why the report can be useful before a failure, we need a few distinctions. In the paper’s terminology, an error is a mistake made by a developer or user. A defect is an issue in the program that can lead to undesired behavior. When that undesired behavior actually occurs, it is a fault. A failure occurs when the fault reaches an invalid state or violates a guarantee of the program.

A code smell is different again. It is a design weakness or a benign violation of a best practice that may make future misuse, maintenance, or failure more likely. A code smell is not necessarily a present defect, and a defect is not necessarily a currently visible failure. These categories form a useful ladder of evidence, not a set of interchangeable names.

That ladder explains the distinctive claim of dynamic defect inference. A traditional test often seeks the final rung: demonstrate the failure. OS-Sanitizer can operate earlier. It may observe a pattern that is not wrong in the current execution but is inconsistent with a safety recommendation, leaves a privilege assumption implicit, or points to a complex region that deserves deeper testing. Its result is valuable when it narrows the search for a latent problem.

The result is also easy to overstate. A heuristic report can be a strong reason to inspect code without being proof that the code is defective. Conversely, a report that turns out to be intended behavior can still reveal an under-tested component or an assumption that future users may misunderstand. The difference between “false alarm” and “useful warning” is often determined by context and by what the investigator was trying to learn.

What this series will ask of the evidence

A sensible reading of OS-Sanitizer keeps three layers separate. First is observation: an application performed an operation, and the system supplied particular surrounding facts. Second is inference: a heuristic judged that the combination resembles a known weakness or code smell. Third is validation: a person or another tool inspected the relevant source, reproduced the behavior, fuzzed the code, or worked with maintainers to decide what should change.

The eBPF subsystem can make the first layer unusually rich by connecting user-space and kernel-space events. OS-Sanitizer’s heuristics operate on that context. But neither eBPF nor the heuristic removes the third layer. The Linux verifier can constrain the safety of an eBPF analysis program; it cannot decide whether the application behavior it observes violates the application’s intended guarantee. Safe execution of the monitor and correctness of the monitor’s conclusion are separate questions.

This separation also changes how effectiveness should be judged. The relevant outcome is not simply a large number of reports. A useful system must expose meaningful code regions, preserve enough context for a person to investigate, and avoid overwhelming that person with repeated or context-free warnings. It must also work at an acceptable cost on the workloads where it is deployed. A method that finds a signal but cannot be run, interpreted, or acted upon has limited practical value.

The supplied study supports a focused conclusion rather than a universal promise. It shows that dynamic defect inference with eBPF is feasible for selected classes of Linux application behavior and that the approach can uncover issues that had remained latent. It does not show that every defect is visible at runtime, that every report is correct, or that eBPF replaces static analysis, ordinary testing, fuzzing, or safer design. Its value lies in covering a gap: finding suspicious behavior before a failure supplies the counterexample.

The next distinction is foundational. If a report is evidence of something that could go wrong, we need to know precisely what separates a defect from the fault and failure that may eventually reveal it—and why that difference matters for interpreting every later signal.

References