Overview

This framework turns public-web-to-model analysis into an auditable decision: state the intended use, map sources and personal-data exposure, identify decision-makers, test lawful basis, necessity, expectations, and impact, then match safeguards to failure points. For private entities using external sources under GDPR-focused EDPB analysis, it supports proceeding with limits, narrowing, redesign, or stopping without resolving every other legal regime.

Learning outcomes

Public access is an input to the decision, not consent or a conclusion that training is permitted.
A defensible decision follows the data from purpose and source through actors, safeguards, model behaviour, and outputs.
Proceeding is conditional; missing evidence, unmanageable impact, or unsupported safeguards can require narrowing, redesign, or stopping.

Keywords

FrameworkDecisionProvenanceSafeguardsAuditabilityGDPR

A Decision Framework for Public Web Data

The final question is whether publicly accessible web material can be treated as free training data for generative AI. The answer supported by the evidence is no—not automatically. Public access is one fact about a source. A defensible use also depends on the purpose, the people and data involved, the organisations making decisions, the lawful basis, the source context, the likely effects, and the controls that continue after training. This framework turns those connected questions into a decision: proceed with limits, narrow the use, redesign it, or stop.

That conclusion is deliberately narrower than a blanket prohibition. It concerns the EDPB’s GDPR-focused analysis of private entities scraping external internet sources or reusing scraped datasets for generative-AI development, especially where personal data are involved. It does not say that every non-personal item on the public web is unusable. It does not resolve every copyright, contract, database-right, platform-policy, or jurisdictional question. Its value is to replace an unreliable shortcut—“anyone can see it, so anyone may train on it”—with a chain of reasons that can be examined.

Public access is an input to the decision, not the decision

The phrase “public web data” compresses several different situations. A freely accessible blog post, an entry in a public directory, a recent discussion page, an old archive, and a social-media post visible only after login may all be reachable under different conditions. They may contain different kinds of personal data, carry different expectations, and create different effects when copied into a dataset and used to shape a model.

The same is true of the phrase “training data.” It may refer to a targeted collection from defined sources, an untargeted crawl that follows discovered links, a cleaned and structured dataset purchased from another organisation, or material that is later annotated, split, validated, and tested. The CNIL schema places collection and preprocessing between system design and learning. The EDPB describes extraction, cleaning, structuring, and storage as processing operations when personal data are involved. A project cannot answer the reuse question until it describes which of these situations it has created.

Public accessibility therefore answers only whether material can be reached in a particular way. It does not establish consent. The EDPB says that people making personal data visible online do not thereby consent to scraping for a specific purpose, and the absence of a robots.txt file is not GDPR consent. Accessibility may still affect reasonable expectations, but it has to be read with the source’s restrictions, audience, publication context, relationship with the controller, and the characteristics of the people represented.

The practical translation is simple: treat “public” as an observation to carry into the analysis, not as the conclusion to carry out of it.

A decision that follows the data

A useful decision framework follows the material from purpose to source, from source to dataset, and from dataset to model and output. Each stage changes what must be known and what can go wrong.

1. State the intended use precisely

Start with the capability or service the project is trying to develop. A purpose such as “make the model better” does not identify which sources or fields are needed. A more precise objective can ask whether the project is developing a conversational agent, detecting fraudulent content or behaviour, improving threat detection, or pursuing a stated general-purpose model objective. The EDPB says the objective of general-purpose development should be described, including whether it is commercial, public, scientific, internal, or external.

Precision is not a ceremonial first paragraph. It is what allows the project to distinguish necessary information from material that is merely available. If the objective concerns technical vocabulary, the project can question whether it needs author names, profile links, precise locations, or unrelated personal discussion. If the objective concerns a defined detection task, it can ask whether a wide archive of personal conversations is needed at all. A purpose that cannot influence inclusion and exclusion rules is not yet doing enough work.

Purpose limitation also follows the data into reuse. A dataset collected for one objective is not automatically suitable for a different model, audience, or capability. The organisation receiving an already-scraped dataset must examine its own purpose and processing, rather than inheriting a blanket authorisation from the earlier collector.

2. Map the source context and the personal-data exposure

Next, identify what the source is and what the project expects to collect. Is the collection targeted or untargeted?

Published by DeepMox

FAQ

What decisions can this public-web data framework produce?

It can support proceeding with limits, narrowing the source or data range, redesigning the use, or stopping when the evidence cannot support it.

When should a project pause or narrow public-web data use?

It should pause or narrow when purpose, actors, source conditions, provenance, rights support, or model safeguards are too uncertain or cannot address the likely impact.

What does this public-web data framework not decide?

It does not settle every copyright, database-right, contract, platform-policy, jurisdictional, non-personal-data, first-party-data, or model-architecture question.

Previous10 minAI Risk Does Not End at Collection

All chaptersBack to Topic