Every business website has at least one very specific job to do.
Your website’s goal might be selling products, generating leads, or maybe just collecting email addresses for your newsletter.
Whatever your goal is, if you’re like the rest of us and would like it to happen as often as possible,… AB testing should be a central part of your marketing strategy.
Most websites will have several KPI’s (Key Performance Indicators) that they track and test.
Regardless of whatever you track, conversion rate optimization (CRO) is widely considered The Holy Grail Of Digital Marketing.
Pearson’s Law: “That which is measured improves. That which is measured and reported improves exponentially.” – Karl Pearson
While many marketers might be concerned with traffic volume, bounce rate, Facebook clicks, or Twitter retweets, the truth is, none of those statistics matter if your conversion rate is not where it needs to be.
There is no better way to test conversion optimization than to put two different versions of your website, head to head, in real-time. And then see which one performs better.
AB testing is the real-time testing of at least two versions of your site, or elements of your site.
Your campaign will consist of at least two separate versions of some part of your website.
You will filter every visitor into one of two segments of visitors. Each of your segments will only see their respective version of your site.
Your campaign will prevent either segment of users from seeing the other version of your site.
The software will analyze the experiment results in real-time to confidently determine, which is the clear winner.
Every visitor to your site proceeds through your pages as usual. They will not realize they are taking part in a critical usability experiment.
During the test, advanced tracking tools will monitor your visitor’s behavior and conversion rates over time, usually some weeks.
The data will be analyzed and monitored until it’s determined that one version of your site performs statistically better, and the results are not likely to change.
Statistical analysis is handled by sophisticated testing software to determine which variation will have the highest impact on your site’s performance.
When a clear winner has been chosen, you can safely make a permanent positive change to your website with a high level of confidence and accuracy.
How does AB Testing work?
Directly comparing multiple versions of your site in real-time is the single best way to improve your conversions.
The Old Days
Historically, testing your website involved making changes to your site and then comparing the success against previous periods.
That method was never reliable.
There is no easy way to predict all the other day to day deviances for starters, let alone week to week influences and fluctuations that may distort the results.
Secondly, there was never an easy way to compare or even discover all of the things that might have made the results different.
An email, a holiday, a technical issue, or a million other items could always skew the results one way or another.
Handling AB testing today.
Today we’re lucky.
We have testing tools that let us run our real-time tests.
The tools will calculate all the statistical data for us. They will continuously recalculate the target number to show a conclusive winner.
The software continues testing and evaluating the data until it reaches statistical significance.
With our winning test results in hand, we can confidently implement our changes to the entire audience or scrap them for the next hypothesis.
The Process
The first step is to analyze data from google analytics or other statistics to find areas where there is an opportunity for improvement.
Your challenge is finding a metric that has room for improvement and then suggest an alternative you think will outperform the current process.
Suggested changes are often personal opinions, industry trends, recommendations from professionals, or solicited customer suggestions.
It’s often beneficial to consult with a digital marketing expert to discover areas where you can improve conversions.
Once you have your test hypothesis ironed out, the software can create an alternate “Variation B” of the element you want to change.
Now, we’re ready to conduct our test.
The software will segment every visitor into their appropriate group once the experiment is activated. Your optimization software will log the test results in real-time.
Your optimization software will log the test results in real-time.
The Technical Stuff
For the sake of speed, businesses usually segment their traffic into two even sections. This approach allows your test to progress much faster, giving your company actionable data as soon as possible.
In some cases, you may be hesitant about your new variant. If you’re worried about distributing this new product feature to half of your customers, you can always lower the target audience size down to as small as 10%.
Statistical significance.
It’s critical when testing to understand the odds. You might flip a quarter ten times and have it come up heads 7 out of 10. That’s 70% good, right!
But, flip that same coin 10,000 times, it would not reach heads even 52% of the time.
Our job is to weigh down the quarter on one side or the other and perform enough tests to hit statistical significance that proves which side is heavier.
Statistical significance is a mathematical equation calculated with tools based on your current conversion rates.
When followed correctly, it will prove which content has a more significant impact on your site performance.
The amount of time and results it takes to reach statistical significance can vary greatly depending on how close the two variations are converting.
To better understand how we use AB testing, let us walk through a hypothetical test together.
First, we must start with a theory that would affect a KPI that we want to increase on the site…
Theory: “Changing the add to cart buttons from the current color of green (this is the original) to orange (this would be the variation) would lead to increased sales on the website.”
We would then target a subgroup of users to the site that would experience this variation.
The “split” could be 50 percent of the traffic seeing the constant and 50 percent of the traffic seeing the variation.
The “experiment” would run on your website for at least two weeks (if not longer). The winner is only declared when there is a statistically significant winner.
A lot goes into this calculation. The results will consider the overall increase (or decrease) in the KPI, the overall site traffic, the number of users, and more.
Let us talk about our hypothetical experiment outcome… Let us say that our variation increased the overall conversion rate from 3% to 3.15%.
At first glance, this does not look like a large number, but we need to break this further down.
The increase in the overall transactions taking place equals a 5% increase (excellent!).
Taking this a step further, if we get 1000 transactions every month, at an average order value (AOV) of $125. Our variation could yield an additional $6,250 of sales per month, and looking at the year. It could be an additional $75,000!
The above hypothesis is a straightforward test. However, the testing capabilities are almost limitless for your website.
Additional testing options include:
Multivariate Testing – Multivariate tests for sites with a substantial amount of traffic and conversions allow you to test multiple elements on a page to see which combination achieves your goals.
Split URL Testing – A redirect test, or split URL test, is a type of A/B test that lets you test separate versions of entire pages against each other, not just elements.
With redirect tests, the test variants are identified by URL instead of a page element, which is most useful when testing two very different landing pages or a complete page redesign.
What Should I AB test?
“Learn the rules like a pro, so you can break them like an artist” ~ Pablo Picasso.
The internet has many established best practices confirmed based on billions of dollars of marketing research by the world’s largest companies.
Building your site around best practice principles in mind is always a good idea. However, every site is different.
If you want to stand out, you have to take a few risks.
These are a few of the things every website should test.
Test anything that breaks the “norms” and or best practices of layout, design, presentation, or marketing.
Call to Action Elements. Examine all buttons or elements on your site that contribute directly to conversions.
Determine if anything is causing friction for your customers, and theorize ways to improve it.
Examine your page and product titles anywhere that contributes to a conversion.
Brainstorm ideas for your home page hero section. It’s the most valuable real estate on your site. Make sure you are utilizing it optimally.
Promotional copy. Are you running a special sale? What will perform better 15% off or Free Shipping?
*AB testing has consistently shown that free shipping outperforms discounts, even when the discounts are of greater value!
Sign up forms. What’s the difference between Subscribe and Sign up? How many fields should your form have?
AB Testing examples
1. Call to Action, placement, and/or design. Does your page layout work better with the Add To Cart button higher or below the details? Does the product price point make a difference?
2. Headlines or category labels or section “Clearance” vs. “On Sale” could there really be a difference? You would be surprised.
3. Product Shot vs. Lifestyle Image – What are my customers more likely to respond to?
4. Hero Section Carousel vs. Section Layout – Is my carousel costing me sales?
5. Promotional Sales – FREE SHIPPING vs. 10% OFF – The results are very clear.
6. Sign Up Forms – More Information vs. More signups
Common AB testing mistakes
- Testing multiple things at once. If you are interested in changing a button’s color and the title on a page, you would want to do these tests in order, not together.
- Making changes during the middle of a test. If you’re testing two different variations, and halfway through you make changes that may affect how the variations are perceived, it’s best to start the test over.
- Unusual or small sample size. A sudden influx of customers due to seasonal events, press, sales, or some other type of event that brings extra traffic to your site can yield inaccurate results. If your testing a specific product that you rarely see results from the sample size of visitors will probably be too small for your campaign to produce results you can use.
- Not setting cookies to maintain a consistent experience. If your customer shows up to your website in the morning and sees Headline A, and later sees headline B you can’t be sure which made them pull the trigger. Setting tracking cookies allows you to maintain a consistent user experience for your visitors.
- Not telling your staff about your experiments. If you’re displaying different versions of products or sales to your customers, your customer service needs to be aware of both versions to prevent some potentially awkward customer service inquiries.
- Finally, by far, the number one mistake people make when running a split test is stopping too soon. Stopping early can be very tempting. Experienced CX professionals will tell you that early leaders will frequently end up trailing by double digits by the time your test reaches statistical significance.
A typical test may require up to 300 conversions to be significant. If you’re not willing or able to spend up to 2-4 weeks running a controlled trial, it’s probably not worth it to run the test.
Final Thoughts
AB Split testing your website, specifically your conversion process, is one of the most valuable marketing technologies that’s come along in the last 20 years.
As useful as this tool is, it loses all value if it is misused. For the best results, consult a conversion optimization specialist.
Rhino Group uses advanced software to diagnose and test optimization opportunities.
If you have a pain point on your website that you would like to review for possible improvements, schedule a meeting with us today, and we can check your site and make proven suggestions to improve your results.
