2 builders who never spoke to each other

Test a belief, not ten variants

Generating ten versions and crowning the best is the same error as quoting a batter's best slice of a season: the winner is unproven, not proven. Write variants from a stated belief about which pain or which wording will grab the reader, then test that belief, rather than producing alternatives out of a thesaurus. Know the conditions that justify a split test at all, millions of impressions, high purchase volume, or a sales cycle short enough that a variant moves revenue this month, and stop behaving as though you had them when you do not.

The method

1. 01

   Treat a winning variant picked out of ten as unproven: generating ten versions and crowning the best is the same error as a commentator quoting a batter's best slice of a season.

2. 02

   Write ad variants from a stated belief about which pain or which wording will grab the reader, and test that belief, rather than generating alternatives from a thesaurus.

3. 03

   Know the conditions that do justify testing: millions of ad impressions, high purchase volume, or a sales cycle short enough that a variant moves revenue this month.

What it returned

• His example of the fallacy is announcing that combination D of ten ad variants is the clear winner.

• SparkToro's line is that a company without that volume or that budget should stop behaving as though it had them.

Sources

• The Pattern-Seeking FallacyJason Cohen · longform.asmartbear.com · The Pattern-Seeking Fallacy · 2010-06-14
• No, I Will Not A/B Test That - SparkToroSparkToro · sparktoro.com · Fine, sometimes A/B testing makes sense. · undated