Every marketing platform can show attractive conversion dashboards. But a message receiving credit for a purchase is not the same as proving that the purchase happened because of the message. This distinction is especially important for brands investing in AI-assisted conversations, follow-ups and omnichannel journeys.
Strong measurement asks a better question: what happened to customers who received a useful interaction compared with similar customers who did not? The answer gives a merchant a more credible basis for deciding which journeys deserve more investment.
Attributed revenue versus incremental revenue
Attributed revenue means a reporting rule assigned a purchase to a message, channel or conversation. For example, a shopper clicks an email and buys within the configured window, so that purchase is counted by the email platform.
Incremental revenue estimates the additional sales caused by the intervention. The shopper might have returned on their own, clicked a paid search ad earlier or had a cart open on another device. Attribution is helpful for tracing activity, but it cannot fully resolve the counterfactual.
Use both measures without confusing them. Attribution helps teams understand observed paths. Incrementality testing helps decide whether a campaign changed behavior.
Build an event model before choosing a dashboard
Decide what a conversion actually means: order placed, paid order, fulfilled order or net revenue after refunds. Agree on which event is authoritative and the time window used for comparing groups.
Maintain a consistent event dictionary covering message eligibility, sending, delivery, clicks, assisted conversations, cart changes, orders, returns and opt-outs. Deduplicate customer identifiers carefully and prevent one purchase from being counted as several separate orders.
Tag channel and campaign origin but avoid assuming the last click caused every conversion. An attribution model is a defined reporting rule, not a complete view of human decision-making.
- Define your primary conversion and refund treatment.
- Record campaign eligibility as well as actual sends.
- Use consistent event names across channels.
- Keep a traceable, documented attribution window.
What is a holdout group?
A holdout is a randomly selected subset of eligible customers who do not receive the specific campaign or journey under test. The treatment group receives it. Because both groups were assigned before messages were sent, the difference in outcomes offers an estimate of incremental impact.
Suppose 1,000 eligible shoppers are randomly divided into a treatment group and a control group. If the treatment group's purchase rate is higher during the same measurement window, the difference can suggest lift—but statistical uncertainty, sufficient sample size and equivalent eligibility still matter.
Do not create the control group by picking customers who never open messages. That would introduce bias because their behavior is already different. Randomize eligible customers before the intervention.
How to calculate conversion lift
Absolute lift in percentage points equals the treatment conversion rate minus the control conversion rate. Relative lift compares that difference with the control conversion rate. Always label which one is being reported.
For an illustrative example, if the treatment group's conversion rate is 6% and the control group's is 5%, absolute lift is 1 percentage point and relative lift is 20%. That example is arithmetic, not an observed Scarvion customer result.
For revenue, compare net revenue per eligible customer or another clearly defined metric. Make sure the groups are similar in size and observation period, and do not confuse revenue per message with revenue per eligible shopper.
Design a fair test for AI conversations
An AI shopping-assistance test might compare customers offered a conversational helper with customers receiving the usual self-service experience. An abandoned-cart journey might compare a permission-based personalized follow-up with a defined existing workflow.
Keep the product catalog, pricing, promotions and relevant policies as stable as practical during the test. Record major external events such as a flash sale, delivery disruption or seasonal peak that could change buyer behavior.
Measure quality alongside conversion: unresolved questions, human escalations, complaints, unsubscribes, refunds and product returns. A journey that converts more purchases but creates substantial dissatisfaction is not an unqualified win.
What Braze and other research illustrate
Braze has published material explaining global control groups and how brands can compare customers exposed to engagement programs with held-out customers. Its retail examples also discuss triggered journeys built around shopping behavior and product availability.
Klaviyo's segmentation guidance emphasizes purchase behavior and careful audience selection. These are external educational references, not independent validation of any Scarvion feature or customer result. The broader lesson is to make attribution definitions explicit and to test whether marketing creates incremental customer value.
Build a weekly decision dashboard
Review eligible customers, delivered messages, assisted resolution, treatment and control conversion rates, net revenue per eligible customer, cost per incremental order, unsubscribes and operational exceptions. Use one primary business outcome per test rather than choosing whichever number looks strongest afterward.
Check confidence intervals or another suitable measure of uncertainty. A small sample can produce a large apparent percentage change that is not stable. When volume is insufficient for a formal test, report directional observations honestly and avoid strong causal claims.
Document decisions: continue, adjust, pause or stop the journey. If the experiment wins, keep a small ongoing control or rerun a fresh comparison after major product and audience changes.
How to calculate incremental e-commerce conversion lift
Quick answer: An attributed purchase is not proof that an AI message caused a purchase. A randomized holdout can estimate the difference between customers who were eligible for a campaign and customers who did not receive it.
- Choose one outcome, such as completed orders from a defined eligible audience, and a fixed observation window.
- Randomly assign comparable eligible shoppers to treatment and holdout groups without excluding inconvenient results.
- Compute group conversion rates using all eligible assigned shoppers, not only message openers.
- Report absolute percentage-point lift, relative lift, uncertainty and contribution margin after message and discount costs.
Practical example and measurement template
Illustrative example, not a Scarvion customer result: Illustrative calculation only: if 50 of 500 treatment shoppers purchase, conversion is 10%. If 40 of 500 control shoppers purchase, conversion is 8%. Absolute lift is 2 percentage points, relative lift is 25%, and the sample may still be too small to establish a reliable difference.
What to measure: Record eligibility criteria, randomization, attribution window, costs, opt-outs and any overlapping promotions. Do not report this hypothetical example as a Scarvion customer result.
Related e-commerce learning:
Frequently asked questions
What is the difference between attribution and incrementality?
Attribution assigns credit to observed conversions according to a rule, while incrementality estimates which conversions were actually caused by an intervention compared with a valid control.
How do you measure AI-driven e-commerce sales lift?
Define an outcome, randomly assign eligible shoppers to treatment and holdout groups, compare conversion or net revenue in the same time window and report uncertainty and customer-experience guardrails.
Is a conversion after an AI chat automatically an AI-generated sale?
No. It may be an AI-influenced or attributed purchase, but proving additional sales generally requires a credible comparison such as an appropriately designed holdout test.
Research and case-study references
These sources document third-party research, examples or platform practices. They are provided for independent reading, not as Scarvion customer results or promises.
Explore Scarvion for e-commerce
See how Scarvion supports online stores with customer engagement, shopping assistance and automation.
Original educational guidance. Workflows are illustrative; performance, integrations and permitted messaging depend on your configuration, approved data and consent. Third-party examples are not Scarvion case studies.