The distinction sounds pedantic until you see the numbers. A chat widget that pops up on every visitor will "touch" a huge share of purchases — including the ones that were always going to happen. Attribution dashboards happily count all of them. A holdout test frequently reveals that the true caused lift is a fraction of the influenced figure.
Influenced vs incremental
| Influenced revenue | Incremental revenue | |
|---|---|---|
| Question answered | "Did the tool touch the sale?" | "Did the tool cause the sale?" |
| Method | Attribution (last-touch, any-touch) | Holdout / control group |
| Bias | Overcounts, sometimes wildly | Unbiased if randomized |
| Vendor incentive | Loves it | Fears it |
How holdout measurement works
- Randomly assign a slice of visitors (say 10%) to a silent holdout — the tool never activates for them.
- Let both groups shop normally over a meaningful window.
- Compare conversion rate and revenue per visitor between groups.
- The difference — and only the difference — is the tool's incremental contribution.
This is the same logic as an A/B test, applied permanently as an honesty mechanism. Yokaify runs continuous holdout measurement for exactly this reason — the methodology is described in measuring impact, and the broader testing discipline in ecommerce A/B testing.
Why this matters when buying conversion tools
Every proactive conversion mascot, chatbot, popup, and personalization vendor will show you a revenue number. The only question that separates real tools from theater: "Is that number from a holdout, or from attribution?" If the vendor can't answer, assume attribution — and assume it's inflated.
Related terms
- Conversion rate — the metric incrementality is measured on
- Cart abandonment rate — where much of the recoverable revenue hides
- Proactive conversion mascot — the category that makes holdout measurement a founding principle
