Read the Signal, Not the Number

OpenSSF Scorecard produces numbers, but a software-vendor decision needs meanings. The relevant question is not whether a repository received a high or low aggregate score; it is which behavior was observed, under what detection conditions, and what uncertainty remains. This chapter shows how to interpret individual scores, weighted aggregation, inconclusive outcomes, coverage limits, and maintainer annotations so that a result can inform a due-diligence decision without pretending to be a universal security verdict.

A score is a compression step. It is useful for finding where attention may be needed, but compression can hide why two projects with the same number have different risk. One project may have a weakness in a high-risk control and strong performance elsewhere. Another may have several modest weaknesses, or simply a low result because its implementation is not recognized by the tool. The number is not wrong merely because it is incomplete. It becomes misleading when the reader forgets what was compressed.

An aggregate is a summary, not a comparator

Scorecard assigns each check a score from 0 to 10. The project documentation identifies the checks as security heuristics and explicitly warns that they are opinionated, with possible false positives and false negatives. It also says the aggregate score tells the reader nothing about which individual behaviors a repository does or does not perform. That is a strong instruction about how to use the output: start with the check and its reason, then inspect the supporting finding; do not start with the aggregate and work backward to a conclusion you already want.

The aggregate is risk-weighted rather than a simple count. In the documented calculation, Critical checks have weight 10, High checks 7.5, Medium checks 5, and Low checks 2.5. A lower score on a higher-risk check can therefore influence the aggregate more than the same score on a lower-risk check. The weights encode Scorecard's view of relative risk, not the acquiring organization's complete risk model.

The denominator matters too. The implementation skips inconclusive check scores when calculating the aggregate. The total weight therefore reflects the checks that produced usable scores, not necessarily every check a reviewer expected to see. The aggregate also changes depending on which checks are run. Two projects can display similar numbers even when they were evaluated with different check sets, and a single project can change its number when an analyst selects a narrower set of checks.

That makes cross-project comparison conditional. A comparison is more meaningful when the repositories were analyzed with the same evaluator version, the same relevant checks, comparable access, comparable repository scope, and similar dates. Even then, the individual mechanisms may matter more than the number. A project with a slightly lower aggregate but a strong control over a buyer's most material threat may be a better fit than a project with a higher aggregate whose weaknesses align with that threat.

A simple hypothetical illustrates the problem. Imagine two libraries with an aggregate of 7.0. Library A scores well on dependency and vulnerability checks but has a serious weakness in workflow permissions. Library B has average scores across several low- and medium-risk checks but does not show that same high-risk workflow exposure. If the buyer's build pipeline is sensitive to workflow compromise, the equal aggregate does not imply equal decision risk. The aggregate has located a conversation; it has not settled it.

The same caution applies to a high score.