Inside the Technical Evidence Layer

A repository-security result is useful to a software-vendor review only when a reader can follow it from an observed signal to the conclusion that was actually drawn. This chapter examines OpenSSF Scorecard as that technical evidence layer: what the run reads, how probe outcomes become findings and check results, and how the result is exported for later review or policy use. The practical capability is not memorizing the project's implementation. It is learning which metadata makes a result traceable and which boundary remains the acquirer's responsibility.

The distinction matters because “the tool gave the project a score” hides several transformations. A hosting service exposes files, settings, history, releases, and other signals. A probe evaluates one kind of question against those signals. A check combines probe results into a reason and a score. An output format then presents some of that information to a person, a code-scanning interface, or a policy system. Each layer can preserve context or discard it. Due diligence becomes stronger when the layers are visible.

From a repository signal to a reviewable finding

Scorecard's central unit is not the aggregate number. Its finding model carries a probe identifier, an outcome, a message, optional values, optional remediation, and an optional location. A location can identify a path, file type, line range, and code or configuration snippet. That structure answers the first review question: what was observed, where was it observed, and what did the analysis say about it?

The probe identifier matters because two observations that both sound like “security failed” may test different mechanisms. One probe may examine whether a workflow uses excessive token permissions. Another may examine whether a dependency is pinned. Their outcomes should not be merged merely because both receive a low check score. Keeping the probe attached preserves the relationship between a finding and the question that generated it.

The outcome vocabulary also carries information. A finding can be True or False, but it can also be NotAvailable, Error, NotSupported, or NotApplicable. Those states are not interchangeable. False means the evaluated question received a negative answer under the probe's rules. NotAvailable means the needed answer could not be retrieved. NotSupported means the implementation or source shape falls outside what the probe handles. NotApplicable means the finding should not be considered in evaluation. An error means the result could not be determined because execution failed.

This is the difference between evidence of absence and absence of evidence. If a probe finds an unpinned dependency in a build workflow, that is a concrete negative observation with a location. If the tool does not recognize the dependency mechanism, a low or missing result says something about coverage, not necessarily about the dependency itself. If the relevant hosting setting requires access the run does not have, the missing observation should remain missing. A downstream reviewer who turns every non-green state into “failed” loses the reason the tool could not answer.

A check result is a higher-level evaluation. In the retained Scorecard implementation, it has a name, version, error field, score, reason, details, and the findings produced by its probes. Ordinary scores range from 0 to 10. Scorecard uses -1 for an inconclusive result when it cannot set a reliable score, including cases in which information is unavailable.