What Privacy Removes from the Signal
This chapter follows the cross-channel advertising measurement system through the reporting layer that stands between a clean randomized experiment and the analyst’s report. The focus is specific: identify what each privacy and identity constraint removes, then decide what the remaining number still means. That distinction lets a team diagnose a weak signal instead of applying one broad “privacy adjustment” to every channel or segment.
A randomized experiment may begin with treatment assignment, control assignment, outcomes, and segment membership. The reporting system may release only partial summaries. The experiment can remain valid in design while the report becomes insufficient for a fine-grained claim. Privacy is therefore not just an extra source of statistical noise. It can change which outcomes are observable, which units can be connected, and which clean effects remain compatible with the released evidence.
From a clean experiment to a degraded report
Start with a clean comparison. For a segment, the incrementality question is the difference between the mean outcome that would occur under treatment and the mean outcome that would occur under control. Random assignment makes that comparison meaningful under the experiment’s assumptions. In a complete record, the analyst could calculate the arm means from the relevant units and compare them.
A privacy-constrained report inserts a sequence of filters and transformations. Some outcomes may not be matchable to the experiment. Some events may not remain linkable to the same person, device, or experimental unit. Some conversions may occur outside the permitted attribution window. Even when an outcome survives those first filters, a reporting service may suppress a small cell or add randomized noise to a released aggregate. The report may therefore contain a valid summary of what the system released without containing a complete record of the original causal comparison.
The first loss is match-rate loss. If an outcome cannot be matched to a reportable unit, it does not enter the released arm mean. Opt-out behavior, missing identifiers, or unmatched conversions can reduce the share of outcome signal that remains visible. This is not equivalent to observing a zero outcome. A missing match says the reporting system did not retain the outcome for this purpose; it does not establish that the underlying unit had no conversion or no value.
The second is linkability loss. A system may observe separate events but fail to connect them to the same identity or experimental unit across time. That breaks continuity in a customer journey and can prevent the analyst from knowing whether an exposure and a conversion belong to the same reportable path. Linkability is different from match rate: an event can exist in the system and still be unusable for the intended longitudinal relationship.
The third is attribution-window loss. A reporting rule may count only conversions inside a defined period after exposure. A conversion outside that window may be real, but it is not part of the reported outcome for that measurement.