Why Scanners Miss What Frameworks Can Execute
A model scanner does not observe execution directly; it constructs an interpretation of an artifact and returns a result. This chapter diagnoses the gap between that interpretation and the framework that will eventually load the model. The key judgment is whether a result means “the analyzed path contained no known issue,” “the tool did not support this path,” or “analysis stopped before a verdict.” That distinction helps readers treat scanner output as evidence with a scope, not as a universal safety certificate.
The previous chapter showed why a framework can reach pickle through several routes. A scanner has to choose its own way to represent those routes, parse them, and decide which behavior deserves attention. Failures can therefore arise before the scanner reaches the risky function, or after it reaches the pickle but understands only a narrow set of functions. These are different causes and require different responses.
A scanner can fail before it judges
The cleanest scanner outcome is not simply a green label. It is a documented result in which the tool recognized the relevant representation, parsed it successfully, applied its checks, and found no issue within those checks. That result is still bounded: it says what the tool found in the path it understood. But it is different from an artifact that the scanner never knew how to open or an input that caused the parser to stop.
Three states should be kept separate:
- Unsupported path: the artifact uses a route the scanner does not model, such as an unrecognized packaging or transformation. The tool may be unable to inspect the relevant pickle at all.
- Scanner exception: the scanner or a dependency encounters an input it cannot handle and terminates before producing a meaningful verdict. The target framework may still accept the same artifact.
- Clean negative: the scanner recognized and analyzed the relevant path and did not find a known problem under its rules. This is evidence about the scan, not proof that no behavior exists.
The distinction is operational, not semantic hair-splitting. If an unsupported path is displayed as clean, the platform has converted missing knowledge into false reassurance. If an exception is swallowed and the upload continues, a crash has become a fail-open security decision. If a clean negative is treated as a proof of safety, the platform has overstated what static rules can establish. A defensible system records these outcomes separately and routes unsupported or failed analysis to a stronger control.
The USENIX study describes a scanner-side version of this problem as Exception-Directed Programming, or EDP. The authors look for exception behavior in scanner code and its parsing dependencies, construct inputs that reach those failure points, and then verify whether a target framework can still load the model. The defensive lesson does not depend on reproducing those inputs.