skip to content

← back to the plays

independence2 unconnected · 1 more author in those circles
standing

contested · 6 claims against 2

applies to

b2b and consumer, no audience needed

confidence

high

evidence

8 claims · 6 articles · 3 receipts

window

2014-09-29 → 2024-08-25

2 builders who never spoke to each other · contested

Test the big change, not the small one

Small variations need far longer to reach significance and, while they run, nothing capable of a double digit gain is being tested. Start with a drastic redesign of the page where the action ought to be obvious, run it against the current version, and stop trying to work out which part of the change caused the result. When one element keeps generating internal debate, test removing it rather than explaining it harder.

Where this splits

Kogan and Haines argue for drastic redesigns tested whole, while Riley works a fixed checklist of levers and starts with the changes that need no design skill at all.

  1. side 01 · 6 claims

    Test one drastic redesign against the current page

    Corey Haines · Greg Kogan

    1. 1Test big changes rather than small ones: small variations need much longer to reach significance and, while they run, you are not testing anything capable of a double-digit improvement.
    2. 2Start conversion work with drastic redesigns of the page where the action should already be obvious, A/B test them against the current version, and save small refinements for later.
    3. 3Stop worrying about which part of a big change caused the result - take the improvement or the lesson and start the next test.
    4. 4Keep quick wins as a side activity: they should not replace the larger tests aimed at the checkout or signup steps that carry the revenue.
    5. 5Incremental split tests only pay off if you have huge traffic and many conversion points, so at low volume make one big change instead.

  2. side 02 · 2 claims

    Work a checklist of small no-design changes first

    Dru Riley

    1. 1Start conversion work with the changes that need no design skill: swap the copy, add an FAQ block, add testimonials.
    2. 2Work a fixed checklist of page levers rather than redesigning: the image, the headline, the proof, the load speed, the call to action and the value proposition.

What it returned

  • $90

    He applied this to a large ecommerce checkout page and says the annual revenue increase was very large, though the numbers are confidential; a comparable checkout he analyzes belongs to a company doing over $90 million in annual revenue.

  • 550

    The calculator debate produced a dedicated Slack channel with over 550 messages and dozens of hours of meetings about what to add; only one person asked whether it was needed at all.

  • He notes the small tests that produce outsized results get written about precisely because they are rare, while the ones producing a half-percent lift go unreported.

Sources

the second source

Three plays a week, for the phase you are in

No roundup of links, no news. Three tactics more than one builder arrived at separately, with the numbers each one returned and the disagreements left in.

One email a week. Unsubscribe in one click, and the address is used for this and nothing else.

More in landing page

all landing page plays

the second source

Three plays a week, for the phase you are in

No roundup of links, no news. Three tactics more than one builder arrived at separately, with the numbers each one returned and the disagreements left in.

963 corroborated plays to draw from · 3 a week

One email a week. Unsubscribe in one click, and the address is used for this and nothing else.