Overview
This series explains why a sale credited to an observable ad touchpoint is not necessarily a sale caused by that advertising. It follows one journey from demand generation through checkout on a generator or trusted marketplace, shows how incomplete recovery can scale a true return into a lower attributed return and distort budget allocation, and compares attribution, MMM, and randomized channel-complete incrementality. It then sets boundaries for ambient experiments, prediction beyond held-out audiences, user-level versus geographic or time assignment, and privacy-degraded reporting. The practical result is a measurement contract: match the claim to its outcome, counterfactual, audience scope, reporting quality, and decision threshold; certify, reject, or leave unresolved when the evidence cannot distinguish the relevant worlds.
Learning outcomes
- Attributed ROAS measures matched revenue within a reporting boundary; it does not by itself identify incremental sales or recover unseen journeys.
- When trust diverts purchases to a marketplace and recovery is incomplete, the generator's measured return and budget can be understated.
- Attribution, MMM, and randomized ITT answer different questions; a causal budget claim needs valid assignment and channel-complete outcomes.
- Prediction can extend experimental evidence only with adequate feature support and re-anchoring, while privacy bounds may leave a threshold decision unresolved.
Keywords
Preview every chapter · Unlock full text, audio, and video from chapter 4
PlansFAQ
What is the difference between attributed ROAS and incremental ROAS?
Attributed ROAS measures matched revenue within a reporting boundary, while incremental ROAS estimates value caused by advertising against a no-ad counterfactual. The two diverge when purchases move to another checkout or signals are lost.
Which measurement method is best for a causal budget decision?
A channel-complete randomized incrementality study is the most direct method for a causal budget decision when assignment is valid and relevant outcomes are covered. Attribution can manage visible response, and MMM can inform aggregate planning when variation supports identification.
Can a randomized advertising experiment still be inconclusive?
Yes, a randomized experiment can remain inconclusive when privacy loss, sparse reporting, or uncertain retention leaves compatible clean effects on both sides of the threshold. The result should be labeled unresolved rather than treated as zero.
When can experimental results be extended to untested audiences?
Experimental results can be extended to untested audiences only when their features overlap the experiment library and the feature-to-effect relationship is stable enough to transport. Fresh randomized activations are needed to check drift.