When Privacy Degrades Even a Randomized Signal

The subject here is the reporting layer that sits between a valid randomized advertising experiment and the decision a brand wants to make from it. A generator's ad may be assigned correctly, and the brand may count purchases on both the generator and the trusted marketplace, yet privacy and identity constraints can still remove or distort the signal. The practical judgment is therefore not just an estimated lift but the strongest threshold claim that survives plausible signal loss: certify, reject, or remain unresolved.

A randomized design can lose information after the fact

Randomization answers a causal question under the experiment's outcome definition. Privacy-preserving reporting answers a different operational question: which parts of that experiment can be released without exposing individuals or violating reporting rules? The second question can weaken the information available to the analyst even when the first design was sound.

Consider the brand's treatment and control groups for an advertising opportunity on G. The clean experiment might contain a conversion or spend outcome for every assigned unit, including a purchase that a treated customer completed on R. The released report may not preserve that clean row-level history. A conversion can be unmatched, the identity link between ad and transaction can break, the purchase can fall outside the reporting window, a small segment can be suppressed, or noise can be added to an aggregate. If those losses differ across customer groups or treatment arms, the observed contrast is no longer a transparent copy of the clean contrast.

The framework groups the reporting problem into six layers:

• Match-rate loss: some outcomes cannot be matched to the relevant experiment or advertising event.
• Linkability loss: identifiers fragment across devices, browsers, or events, breaking continuity between exposure and outcome.
• Attribution-window loss: conversions outside the reporting window are censored even if they occurred later in the journey.
• Aggregation-threshold suppression: sparse cells are withheld when their counts fall below a minimum reporting threshold.
• Randomized reporting noise: released totals or means are perturbed to reduce disclosure risk.
• Segment-heterogeneous signal loss: retention and reporting quality vary across channels, customer histories, or other groups.

These layers can overlap. A marketplace purchase may be difficult to match, the customer identifier may not persist into the transaction, and the transaction may fall outside the chosen window. The framework does not require the analyst to pretend that one exact loss mechanism is known. It represents a collection of plausible mechanisms in an ambiguity set, U. An element of U can specify arm- or segment-specific match probabilities, linkability and window retention, aggregation thresholds, noise scale, and heterogeneous-loss behavior.

The ambiguity set is a statement about what the analyst is willing to defend. It can be informed by reporting documentation, data-quality diagnostics, product knowledge, or stress tests. If the set is too narrow, the resulting certificate may be optimistic because it rules out signal-loss mechanisms that could be present.