Do small wins compound? Six measured experiments, one revenue per visitor discipline at Evans
Overview
A portfolio of small bets across the full journey, with each experience individually measured on RPV and each mapping regional and behavioral data, together contributing several million pounds of top line revenue.
Personalization & Experimentation Specialist
Capita · Evans
Portfolio testing, RPV validation, behavioral urgency
2.5% to 8.43% RPV uplift
A portfolio of small bets, each one measured
Evans posed a different question than its sister brands: not 'what is the one big fix?' but 'what do many small, individually measured improvements add up to?' The program ran a portfolio of six distinct experiences across the product page, basket, mobile, onboarding, and checkout, each proving its own revenue per visitor uplift, and each grounded in behavioral data rather than marketing assertion.
Embedded in the Arcadia ecosystem, I worked on experiences serving a customer base that was predominantly mobile, which made simplicity, honesty, and orientation design requirements, not niceties.
01: The Tension
Three constraints that shaped the program
1: Conversion rate was not enough
A conversion only scorecard can flatter interventions that convert more people into smaller baskets. For a brand like Evans, a single clear metric was needed: revenue per visitor, a single number that holds every experience accountable for the whole commercial outcome. Choosing it was the program's first analytical decision, made before any experience shipped.
2: An audience that punishes complexity
Evans' predominant customer base was not digitally native. Cluttered interfaces, aggressive popups, and manipulative countdowns do not just underperform with this audience; they break trust. Every urgency and proof mechanic had to be simple, legible, and factually true.
3: The big bang temptation
The alternative path, which is one dramatic redesign, concentrates risk and defines clean measurement. A portfolio of small experiences meant each idea could carry its own business case, its own test, and its own numbers. Small, safe, honestly scored.
This program's key wouldn't come from one heroic change. It would come from many modest, measured, data grounded improvements — and from a metric strict enough to keep them all honest.
02: The Craft
The experiment portfolio
Six experiences across four layers of the journey: every mechanic grounded in live behavioral data, every experience individually scored on RPV. Click a bar to explore each experiment.
Popularity messaging on top-200 products
Popularity messaging on products in the top 200 added to bag over the previous 24 hours. Behavioral data is persuasive: local, recent, and true.
03: Rigorous Execution
Every experience in the portfolio was measured individually on RPV. There was no blended program number hiding weak performers behind strong ones. An experience earned its place with its own uplift or it did not stay.
The urgency mechanics were factual by construction: 'selling fast' and popularity flags derived from the trailing 24 hour added to bag data. If a product was not actually moving, it carried no badge. For an audience that punishes manipulation, honest scarcity was not just ethics; it was a retention strategy.
And the portfolio math was treated with respect: RPV uplifts do not naively sum. Experiences overlap in audience and surface, and effects interact. The program's several million pound contribution was assessed at the revenue line, not by adding percentages, a discipline that keeps a portfolio's story credible.
04: Measuring the Lift
best single uplift: popularity messaging on the top-200 added to bag items.
experiences individually measured: each carried its own RPV uplift, from 2.5% to 8.43%.
several million pounds added to top line performance across the personalization program.
product page, basket, mobile, home page onboarding, and checkout: persuasion where it helps, simplification where it counts.
The Arcadia measurement standard applied here as everywhere: digital investments are evaluated on incremental value — engagement, loyalty, and revenue that would not have happened anyway.
Key learnings
Portfolios beat big bangs
Many modest, measured wins compounded into millions, with less risk, faster learning, and cleaner attribution than any single redesign could deliver. The deeper lesson: an experimentation program is not just about code; it is about team alignment, governance, and cadence.
The metric is the strategy
Focusing on a single metric (RPV) changed what winning meant for every experience. Conversion rate can rise while revenue falls; RPV catches both. Metric selection is reporting detail; made early, it shapes design thinking and keeps execution honest.
Uplifts do not sum
The naive math of this portfolio, which is to add up percentages and claim a 25% total lift, is wrong. Knowing why it is wrong, due to overlapping audiences, interacting effects, and different exposure denominators, matters more than any single result. Honest aggregation is where analytical credibility is won or lost.