skip to content

← back to the plays

independence2 unconnected · 3 more authors in those circles
standing

contested · 7 claims against 3

applies to

b2b and consumer, no audience needed

mixed results

confidence

high

evidence

13 claims · 11 articles · 4 receipts

window

2012-11-12 → 2020-07-01

2 builders who never spoke to each other · contested

Read the price test properly before acting on it

Revenue per visitor, not conversion rate or average sale alone, is the number that settles whether a price move worked, and it takes months to read at low volume. A change that looks like a win inside the first weeks can surface later as churn on the upper tiers, so measure through to lifetime value and hold the new number long enough for it to become normal. There is a real split on machinery: one camp builds split tests and flags the change to half the traffic, the other says at this size the answer is obvious without statistics and the setup cost is wasted.

What both sides agree on

  • At roughly ten sales a month you cannot read a pricing change from monthly revenue, so hold the new price for several months and resist reversing it on one bad month.
  • Hold a new price steady long enough to become the norm before touching it again.
  • Expect price optimization to deliver a fraction of the lift you forecast; it moves revenue but rarely doubles it on its own.

Where this splits

Barry and Zue run pricing changes as measured experiments with flags and A/B tests, while Tringas argues split testing prices at low traffic returns nothing you could not have guessed.

  1. side 01 · 7 claims

    Split test the price change and read the data

    Cory Zue · Harry Dry · Justin Jackson · Nathan Barry

    1. 1Judge a pricing test on revenue per visitor, which is the purchase rate multiplied by the average sale price, rather than on either number alone.
    2. 2Watch conversion rate rather than revenue when you push prices, and roll a price rise back when conversion falls.
    3. 3Run a price test for at least 60 days and measure it through to churn, downgrades and lifetime value, since a change that looks like a revenue win at first can end up net negative.
    4. 4A/B test the pricing page change rather than implementing the advice blind, because what works on one product will not on another
    5. 5Do not take a general answer on whether to run a free trial; run it as a test on your own product.

  2. side 02 · 3 claims

    Skip the split test, just change the price manually

    Tyler Tringas

    1. 1Accept some extra churn as the price of a price rise, because the customers who remain generally more than cover the ones who leave.
    2. 2Do not attempt formal price segmentation experiments at low traffic; you will spend the effort and find the money lands on the couple of prices you would have guessed anyway.
    3. 3Do not build split testing for prices: a higher price can look better and then show up as extra churn later, and in a business this size any change worth making is obvious without statistics.

What it returned

  • $285

    After a January of about $285, his lowest since October, with a large drop in sales conversion, Zue returned prices to their December levels to try to recover the November and December conversion rate

  • 50%

    Zue raised the Pegasus license 50% and found the following month's decline uninterpretable, partly because the prior month had been inflated by promoting his Django Stripe guide.

  • 50%

    Zue predicted price work would double revenue and got roughly a 50% increase over eight months.

  • Plotting revenue per visitor against price showed higher prices generally beat lower ones, though the data stayed too noisy to name an optimum, with a December holiday point skewing the curve.

Sources

the second source

Three plays a week, for the phase you are in

No roundup of links, no news. Three tactics more than one builder arrived at separately, with the numbers each one returned and the disagreements left in.

One email a week. Unsubscribe in one click, and the address is used for this and nothing else.

More in pricing

all pricing plays

the second source

Three plays a week, for the phase you are in

No roundup of links, no news. Three tactics more than one builder arrived at separately, with the numbers each one returned and the disagreements left in.

963 corroborated plays to draw from · 3 a week

One email a week. Unsubscribe in one click, and the address is used for this and nothing else.