A/B split testing is the scientific way to see if a change is helping or hurting your site. Or if the change is just a change. It is founded in the core belief that trial and error economics works far better than HiPPO marketing.
How does A/B testing work?
The idea is to change some part of your website and see if the new version does better than the old. A good place to start is just changing words – maybe a headline. You call the existing version “A” and the new version “B“.
You already know that the page is converting at 2% so you could just try the new headline for a few days and check the new conversion rate.
Trouble with this method is that conversion rates change from day to day. You can guess at some of the reasons: weather, day-of-week, competitor activity. So you may be trying a bad headline on a good day or a great headline on a lousy day – and make the wrong decision about the headline.
Focus on what has changed
We can remove these other factors from the test. We show A and B at the same time. Software randomly decides if a new visitor to the site is going to see A or see B, then tracks visitors through to conversion and measures if A or B is doing better.
- If it’s raining – people will see both A and B.
- If it’s sunny – people will see both A and B.
- If a rival starts a price war – people will see both A and B.
You have removed the other factors.
Is it significant?
We use a statistical test to check that results are significant – and not just random effects. Sometimes improvements are small, but still worth having. Several 2-3% increases start to add up.
Is it just Connected doing split testing?
Amazon, Zappos, Google, eBay, EasyJet – all the big names use A/B split testing. Join the big boys – with our help.
