The Completeness Reality Check

An AI Bill of Materials (AIBOM) can be generated from a public model repository, but generation is not the same as evidence completeness. This chapter examines what a large-scale Hugging Face study actually measured: how much repository information survives conversion into a CycloneDX-based AIBOM, which fields are present or missing, and how documentation signals cluster. The practical judgment is whether a valid artifact answers the questions a downstream user has about reuse, limitations, safety, licensing, and provenance—not whether the artifact merely opens in a parser.

The distinction matters because an automated generator cannot recover information that a repository never supplied. It can organise a model card, metadata, links, and configuration into a machine-readable form. It can also preserve the absence of those things. A score may then reveal a documentation gap in the supply chain rather than a defect in the JSON format. That is exactly why the study is a useful complement to an operational AIBOM design: one asks how provenance can be captured, while this evidence asks how much useful material is available to capture in the first place.

What the HF study measured

The study began with a snapshot of 2,942,466 public Hugging Face model records. It retained 97,940 models with more than 100 downloads for AIBOM generation and analysis. The filter was intended to keep the work computationally feasible and focus on repositories with observable reuse. It also shapes the result. Low-download models, private or gated repositories, enterprise registries, and models hosted outside Hugging Face are not represented by the analysed set.

For each retained model, the researchers used the OWASP GenAI Security Project AIBOM Generator. The generator accepts a Hugging Face model identifier, gathers available repository metadata and model-card content, and organises it into a CycloneDX-based machine-readable artifact. The study then validated whether the result was parseable and contained expected structural fields before using it in the completeness dataset. Missing, incomplete, and inconsistent repository information was not manually repaired. That choice is central: the experiment measures the documentation that the source repositories make available to an automated process, not what a researcher could infer by filling gaps from outside knowledge.

The study's completeness score comes from the generator's existing scoring method; the researchers did not define a new metric. It assigns points across five categories. Required fields, metadata, and component-basic information each contribute 20 points. Model-card documentation contributes 30 points, and external references contribute 10 points. The final score is based on the proportion of fields present in each category, with tool-defined penalties for missing critical or important fields.

This construction makes the score useful for one purpose and unsafe for another. It can compare field coverage within the analysed artifacts. It cannot serve as a universal readiness threshold, and it does not measure model performance, fairness, safety, or suitability for a particular application. A model with a higher score may simply expose more of the fields the generator rewards. The study therefore combines the score with field-level parsing and a diagnostic check for meaningful descriptions.

flowchart LR A["Public HF model snapshot"] --> B["Keep models with more than 100 downloads"] B --> C["97,940 retained repositories"] C --> D["OWASP AIBOM Generator"] D --> E["CycloneDX-based AIBOM artifacts"] E --> F["Parse and validate structure"] F --> G["Overall and category scores"] F --> H["Field presence and meaningful-content checks"] F --> I["Repository-group comparisons"] G --> J["Bounded documentation-readiness judgment"] H --> J I --> J

The measurement path is therefore narrower than the phrase “AI transparency” might suggest. It begins with public repository material, passes through one generator and one scoring method, and ends with documentation coverage in valid generated artifacts. It does not inspect the truth of a model card's claims, test the model in deployment, or prove that a missing field means the underlying information does not exist elsewhere. The study is strongest when read as an empirical audit of machine-readable documentation readiness under these defined conditions.