Incrementality

What incrementality means

Incrementality answers the only question that ultimately matters for a marketing dollar: did this spend cause a sale that would not have happened otherwise? A channel can be credited with thousands of conversions by an attribution model and still be barely incremental, because many of those buyers were going to purchase anyway. Incrementality strips out that "would-have-happened" demand and measures only the lift the channel genuinely produced.

The contrast with attribution is the whole point. Attribution is a bookkeeping exercise — it divides credit for conversions among the touchpoints that preceded them. Incrementality is a causal exercise — it compares what happened with the ad to what would have happened without it. The two routinely disagree, sometimes by a wide margin.

How you measure it: the holdout experiment

The gold standard is a randomized holdout. You withhold a channel (or a campaign) from a randomly chosen group and keep running it for everyone else, then compare conversion rates between the two groups. The difference is the incremental effect. Two common designs:

• Geo-lift test: pause the channel in a set of matched geographic markets (the holdout) while it keeps running in comparable control markets. The revenue gap between matched geos is the lift attributable to the channel.
• Audience / cohort holdout: randomly exclude a slice of your audience from a campaign and compare their conversion rate to the exposed group.

From a holdout you can compute incremental ROAS (iROAS) = incremental revenue ÷ spend on the tested channel — the return that survives the "would-have-happened" filter.

Why incrementality and attribution disagree

Three structural reasons:

1. Branded and retargeting demand. Ads served to people already heading to checkout (brand search, cart retargeting) get full attribution credit but produce little incremental lift.
2. View-through and long windows. Long attribution windows credit a channel for conversions it may have only grazed.
3. Overlap. When several channels touch the same buyer, each platform claims the conversion in its own dashboard, so summed attributed revenue exceeds real revenue — but incrementality is measured against one true revenue baseline.

Illustrative example

A brand's attribution model credits its retargeting campaign with a 9.0x ROAS. A geo-holdout test pauses retargeting in half the markets for three weeks. Revenue in the holdout markets barely moves relative to control — the people being retargeted were buying regardless. The campaign's incremental ROAS turns out to be far lower than its attributed 9.0x. The brand redeploys most of that budget into prospecting, which the same test shows is doing the real acquisition work. The numbers here are an illustrative scenario; the recurring pattern — retargeting over-credited, prospecting under-credited — is the lesson.

How to run incrementality well

1. Pick matched markets or a clean random split. The holdout and control must be comparable, or the "lift" is just noise.
2. Run long enough to clear the conversion lag for your category (see conversion-lag windows by channel).
3. Test one variable at a time so the revenue delta is attributable to the channel you paused.
4. Treat observational "incrementality" as directional only. Dashboards that estimate lift without a holdout can flag candidates worth testing, but only a randomized experiment yields a causal number.

Common mistakes

• Calling attribution "incrementality." A multi-touch model is still bookkeeping; it does not establish causation.
• Holdouts too small or too short to detect lift above natural variance.
• Stopping a test the moment results look bad. Early reads are noisy; pre-commit to the window.

How Admaxxer measures incrementality

Admaxxer supports geo-lift and cohort holdout testing and reports incremental revenue against your actual store revenue, so you can see the gap between what a channel is attributed and what it actually caused. Paired with blended MER as the always-on health metric, holdout tests become a periodic truth-check on where your budget is really earning its keep.