Are these hero and CTA benchmarks measured A/B lift or just observed changes?
- By Ali Abouelatta
- n = 2,160
- Published
- Updated
- 2 min read
- landing-page · experiments · ux-patterns · web · mobile
Short answer
They are observed changes, not measured lift: all 488 hero and 232 CTA figures come from detected before/after UI diffs with model-inferred rationale, drawn from 2,160 annotated experiments.source[1] The impact and confidence scores are model-assigned 1-5 values usable only for relative ranking.source[2] Read every stat as 'what changed and the likely reason,' never as a proven conversion result.
Hero/CTA stats are detected UI diffs with inferred rationale, not measured lift — 2,160 annotated experiments, July 2026.
Bar chart · Avg impact (1-5)
Avg impact (1-5) — Breakdown: how to read the scores
One block is about 0.1 Avg impact (1-5)
- FORM
- 3.84
- OFFER
- 3.69
- PRICING
- 3.52
- HERO
- 3.28
- CTA
- 3.16
Finding: detected, not measured
The experiment corpus captures before/after UI changes and an LLM's inferred rationale for each. It does not contain a measured conversion delta.source[1] So a statement like 'Cycle changed its CTA from demo to free trial' is verifiable from the diff; 'and it lifted signups 12%' is not something this data can say.
Breakdown: how to read the scores
Impact and confidence are model-assigned 1-5 scores, meaningful only for relative ranking at N>=20.source[2]
| Area | Avg impact (1-5) |
|---|---|
| FORM | 3.84 |
| OFFER | 3.69 |
| PRICING | 3.52 |
| HERO | 3.28 |
| CTA | 3.16 |
A 3.84 form score does not mean 'forms convert 3.84x' — it means the model rated form changes as relatively higher-impact than CTA changes in this set.
How to apply
Use these benchmarks to prioritize what to test and to see how peers frame changes — not to skip your own experiment. Keyword-incidence theme counts are non-exclusive ILIKE matches and are lower bounds. Every downstream page in this family carries the same caveat: observed change plus inferred rationale, validated by your own test.
The numbers
| # | Stat | Computed from |
|---|---|---|
| [1] | 2,160 annotated experiments (of 4,814 detected); hero 488, CTA 232 are before/after diffs with inferred rationale | hero_experiments_total / cta_experiments_total / smallSampleWarnings[detected not lift] |
| [2] | Impact/confidence are model-assigned 1-5 scores (FORM 3.84, OFFER 3.69, PRICING 3.52, HERO 3.28, CTA 3.16) | experiment_area_leaderboard |
Methodology
Universe: 2,160 annotated experiments, July 2026. Method: before/after UI-diff annotation with model-inferred rationale and 1-5 impact/confidence scores. Caveat: no measured conversion lift; scores rank relatively; keyword themes are non-exclusive lower bounds.
Sources
- [1]
Lazyweb Research analysis of 2,160 annotated experiments (detected-experiment corpus), July 2026
Detected before/after UI diffs with LLM-inferred rationale; not measured A/B lift.
- [2]
Lazyweb Research analysis of 2,160 annotated experiments (detected-experiment corpus), July 2026
Impact/confidence are model-assigned 1-5 scores for relative ranking only at N>=20.
Source: Lazyweb Research — proprietary analysis of real, in-market app screens. Cite as Lazyweb Research, 2026-07-07.