Why a Vendor's Questionnaire Is Not Its Evidence
Software-vendor due diligence is an investigation into a particular supplier and product, not a ceremony for collecting completed forms. The practical question is what an acquirer can independently observe, verify, and connect to a decision about that software. This opening establishes an evidence-first model: use public technical and organizational signals to build a grounded picture, then use supplier questions to close material gaps rather than to stand in for investigation.
That distinction matters because a supplier's answer and a verified finding are different kinds of information. A vendor may accurately describe a control, but the description is still a claim until its scope, operation, date, and supporting evidence are checked. Conversely, the absence of an observable signal is not automatically proof that the underlying practice does not exist. Good diligence has to preserve both facts: what is visible and what remains unknown.
A completed form is still a set of claims
NIST's Cybersecurity Supply Chain Risk Management Due Diligence Quick-Start Guide defines due diligence as researching and verifying all available, pertinent information about a supplier or product so that an acquirer can make an informed decision. It calls this the minimum reasonable research for most suppliers, prioritized by criticality, before a fuller supplier review. That sequence reverses a common procurement habit. Instead of beginning with a long form and treating the answers as the baseline record, the acquirer first establishes what can be learned without asking the supplier to describe itself.
NIST calls this first mode basic due diligence: desktop research using publicly available information. Enhanced due diligence can add commercial datasets, proprietary sources, and supply-chain illumination tools. The distinction is not between serious diligence and careless diligence. It is a way to match the cost and depth of research to the importance of the acquisition. NIST also recommends validating findings against multiple sources when possible. A public record may be incomplete, stale, or referring to a similarly named organization; corroboration is part of the work, not a decorative footnote.
The result should be a set of findings with an object, a source, and a boundary. The object is the legal supplier, the product, the relevant version or service, and the relationship to the acquiring organization. The source says where the observation came from and when it was collected. The boundary says what the observation supports and what it cannot establish. A “yes” in a questionnaire usually compresses all three into one answer. That can be convenient, but it makes it harder to tell whether a statement applies to the contracted product, to the supplier's entire organization, or merely to an aspiration.
A technical result can be misattached in the same way. A repository may be a development project, an upstream component, a fork, or an example rather than the artifact the buyer will deploy. A strong signal from that repository may still be useful, but it cannot authenticate the supplier's legal identity or prove that the reviewed source produced the purchased release. Before any score is interpreted, the acquirer needs to know what entity and product the evidence actually describes.
The diagram captures the central change in sequence. Questions still appear, but they appear after the acquirer knows what is missing and why the missing information could change the decision. That is more demanding than asking every supplier the same set of questions, but it is also more efficient: a low-criticality purchase may need only a public first pass, while a critical product may justify enhanced research, a supplier review, or a system-specific assessment.
Due diligence is wider than software hygiene
NIST organizes ICT-supplier research into five connected categories. They are not five independent boxes to mark complete; together they describe the supplier, the product, and the relationships around both.
Foreign Ownership, Control, or Influence asks who can affect the supplier's management or operations. NIST points to ownership, investment, headquarters, leadership ties, impending transactions, and laws that may require cooperation with foreign security services. Those laws can reach sensitive information such as source code, designs, or customer data. None of this is a code-quality property. A well-managed repository cannot, by itself, resolve a question about beneficial ownership, jurisdiction, or legal influence. The organization must define which countries and levels of foreign exposure matter for its systems and how such findings can be mitigated.
Provenance follows the origin, development, ownership, location, and changes of a system or component, including its subcomponents. For software, that includes open-source and third-party code, proprietary development, vendors, and resellers. An inventory or software bill of materials can make relationships visible, but NIST is explicit that an SBOM does not automatically make software secure. It enables a more tailored assessment. A claim about origin becomes stronger when it is connected to source history, release artifacts, and evidence that the artifact was produced as described.
Resilience asks whether the supplier can keep meeting its obligations and whether the product can remain reliable and trustworthy. NIST includes financial distress, legal and regulatory problems, leadership turnover, breaches, ransomware, environmental instability, poor product performance, counterfeiting, and repeated negligence among possible concerns. These are not all cybersecurity controls. They are reasons a supplier may fail to deliver, support, or protect what the contract depends on. A repository scan can inform one part of the cyber picture, but it cannot establish solvency, manufacturing quality, lawful conduct, or service continuity.
Foundational Cyber Practices has two sides. One concerns the supplier's information-technology assets and public-facing posture; the other concerns product development and product-specific issues such as vulnerabilities, update cadence, end-of-life, and component risk. This is the category in which a software-security tool can contribute the most. Even here, the distinction between supplier IT and the product's development process matters. Evidence that a project protects its source changes is not automatically evidence about every network, service, employee, or release operation belonging to the supplier.
Supply Chain Tiers places suppliers in relation to the end user. A first-tier supplier serves the end user directly; a second-tier supplier serves the first tier, and the chain can continue to a depth the organization chooses. Going farther can reveal common sub-tier dependencies or a sole-source concentration, but NIST warns that the difficulty and cost increase sharply. The right stopping point depends on criticality and risk tolerance. The goal is not to enumerate every distant relationship regardless of consequence; it is to find the relationships that could change the decision.
This wider map explains why “the vendor passed its security questionnaire” is too small a conclusion. The answer may address a control, but it may not address the identity of the contracting entity, the origin of a component, a shared sub-tier dependency, the supplier's ability to continue service, or a relevant foreign relationship. Evidence-first diligence does not discard the answer. It places the answer in the category where it belongs and asks what would corroborate it.
Scorecard is an evidence layer, not the whole supplier
OpenSSF Scorecard is a useful example of what automation can contribute. Its stated purpose is to help open-source maintainers improve security practices and help consumers judge the risks their dependencies introduce. It evaluates selected security heuristics, assigns individual checks scores from 0 to 10, and can support decisions about accepting risk, considering alternatives, or working with maintainers.
The important word is selected. Scorecard's own project documentation says that it is not a definitive report or a one-size-fits-all requirement. Its checks are opinionated heuristics and can produce false positives and false negatives. The aggregate score is a risk-weighted summary; it does not tell the reader which individual behaviors produced that result. A high aggregate number therefore cannot answer whether a supplier is acceptable in every context, just as a low number cannot prove that the software is unusable. The number is a way to navigate the evidence, not a replacement for reading it.
The individual checks are more useful because they connect observations to mechanisms. Branch Protection examines rules such as required reviews, status checks, and restrictions on force-pushing or deleting important branches. Pinned-Dependencies looks for dependencies in build and release paths that are not fixed to a specific hash. Vulnerabilities uses known vulnerability data. Signed-Releases looks for signature or provenance assets. These are meaningful questions because each corresponds to a failure surface: unauthorized code changes, mutable inputs, known flaws, or unverified release artifacts.
They are still bounded observations. Scorecard's documentation says that its dependency-update check can detect whether Dependabot or Renovate is enabled, but not whether the tool actually runs or whether its pull requests are merged. Its signed-release check recognizes relevant asset names but does not verify the signatures. Other checks can miss alternative tools or implementations, and some depend on the hosting platform, a recent-commit window, or access to administrative settings. A result must therefore be read as “this signal was or was not observed under these detection conditions,” not as “the supplier does or does not practice security.”
That is where a questionnaire becomes useful again—but only as a targeted instrument. If an automated result cannot detect a control because the supplier uses another tool, the next question can ask for the implementation and a verification artifact. If a release appears to include a signature, the question can request the verification method and the relationship between the signed artifact, the source commit, and the product being purchased. If the technical evidence is strong but ownership, resilience, or foreign influence remains unclear, the questions should address those gaps instead of repeating repository-security items.
The acquirer remains responsible for connecting the pieces. Scorecard can illuminate selected Foundational Cyber Practices and parts of software provenance. NIST supplies the broader categories and the discipline of criticality, corroboration, risk tolerance, classification, and refresh. Supplier answers can add context, explain an unsupported implementation, or provide evidence that is not public. None of those sources should silently inherit the authority of another.
The first conclusion is consequently modest but practical: begin with evidence that can be observed and checked, preserve what it does not show, and let the remaining uncertainty determine the next question. The next task is not to score a name on a form. It is to identify the supplier and product precisely, decide how much scrutiny their criticality warrants, and set the boundaries within which every later technical signal will be interpreted.