4 builders who never spoke to each other

Run one experiment at a time and price it first

Count the full cost of a test before agreeing to it, scoping, building, QA, tracking and weeks of runtime you could have spent shipping, then run few enough at once that the results are readable. Point them at the flow where money actually changes hands, hold one number as the arbiter so you can accept a decline elsewhere, and send every experiment to the same destination so the comparison is clean.

The method

1. 01

   Price a test properly before you agree to it: scoping, building the variant, QA, tracking, monitoring, then weeks of runtime you could have spent shipping the next campaign.

2. 02

   Work one customer profile at a time rather than running experiments at all of them at once, because parallel efforts across segments make it impossible to tell which one lifted signups.

3. 03

   Run at most one or two channel experiments concurrently, since SEO and paid search both take months to report back.

4. 04

   Send every promotion experiment to the same destination, otherwise each one costs you a fresh decision about where to point people and the results never tell you cleanly what worked.

5. 05

   Point a year of experiments at the single flow where money actually changes hands rather than spreading tests across the whole product.

What it returned

• $3

  Twelve months of tests on one checkout flow added over $3 million a year in extra donations for GiveDirectly, enough to fund roughly 3,000 more families annually.

• SparkToro frames the opportunity cost as the real number, since even a simple test needs someone to scope, build, check and watch it before significance arrives.

• Podscan serves four or five profiles: marketers, PR agencies, founders in media and podcasting, news agencies and SEO professionals

• Splitting attention across three properties sometimes left him unable to choose a destination at all, so he ran nothing.

Sources

• 🧠 Focus – Jakob GreenfeldJakob Greenfeld · jakobgreenfeld.com · 🧠 Focus · 2022-05-02
• Ten SaaS marketing principles for 2024Justin Jackson · justinjackson.ca · 5. Focus on 1-2 marketing experiments at a time · undated
• No, I Will Not A/B Test That - SparkToroSparkToro · sparktoro.com · undated
• Balancing Multiple ICPs: Strategies for Solo FoundersArvid Kahl · thebootstrappedfounder.com · The Time-Boxing Strategy · 2025-09-19
• How GiveDirectly increased donations by over $3 million/year through experimentationOlivia Chen · lennysnewsletter.com · Wins, misses, and lessons from GiveDirectly's donation flow · 2024-12-03
• The system I use to be consistentLuca Restagno · solopreneurtofreedom.com · Personal updates · 2024-03-03