A/B Testing: What It Is and How to Do It Right

A/B testing can be used to see which ads people are engaging with, what buttons are getting more clicks, and which headlines are drawing people in.

No one likes to waste time or money on a web experience that doesn’t perform and convert. As marketers, it’s our job to determine what our audience wants to see and how they want to engage with us online. That’s where A/B testing comes in.

What is A/B testing?

A/B testing (often called split testing or A/B/n testing because you can have “n” variations as opposed to just two) allows you to compare different website experiences to see which one is resonating better with your visitors and will help you achieve the ultimate goal for your site, whether it’s revenue, email signups, ad clicks, etc.

This type of testing splits your web traffic randomly and evenly between your existing web page (the baseline or “Control”) and one or more different variations of the same page.
For example: visitor 1 sees the Control, while visitor 2 lands on web page variation A and visitor 3 on variation B.

With A/B testing for a digital experience or website, content, text/copy, headlines, menus, buttons (text and color), layout, images, or even design can be varied. The goal is to create a page that beats the conversion rate of the Control variation and a minimum of two variations are usually tested against the Control. For successful A/B test, a baseline amount of web traffic (typically thousands of unique visitors per month) is needed to get valid test results.

Why is A/B Testing Important

A/B testing can occasionally be overlooked because it does take time and that’s not a luxury many people have when you’re trying to move at the speed of the web. However, the advantages of A/B testing, which include better conversion and increased revenue, greatly outweigh the additional investment of time.

The more you test different web experiences, the better you will understand the traffic coming to your site. This can be especially important if you’re also running paid traffic initiatives. Typically, paid traffic does not have high conversion, so you’ll want to maximize the return on any additional spend. You’d be surprised at how much impact small website tweaks can have if they are based on testing.

Getting Started

Deciding what you want to test and what your overall goals are is a good place to start. A/B testing can be used to see which ads people are engaging with, what buttons are getting more clicks, and which headlines are drawing people in.

According to a test by MECLABS, something as simple as the call-to-action (CTA) wording on a button can affect the click-thru rate up to 26%.

Katelyn Fogarty, Acquia’s Sr. Manager of Digital Marketing, mentioned in her post Website Testing: What you need to know about A/B Testing, the best elements to start with are the following:

  • Buttons: With buttons, look into testing not just text but also position, size and color of your buttons to see what combination makes your visitors click it.
  • Content: We all know that content is key to the success of any website so why not start off your testing with it? This can include anything from headlines to articles, from product descriptions to photos and videos. What are your visitors engaging with? When you test content, look for what is increasing time on site, pageviews, conversion and revenue, not just clicks.
  • Forms: Forms are an excellent way to turn anonymous visitors into identified visitors. As we mentioned earlier, test out different text on the form or in the CTA to fill it out, different locations of the form on your site, or even things like having the form appear in a lightbox.

Tips for A/B Testing

A/B testing is a valuable practice for making sure you reach your website goals. Here are some tips to make the most of tests:

As Kate points out, make sure you have enough testing data before deciding which variation performed best! The more time to gather information, the better. If your test is not working the way you want it to or the information is not as concrete as you’d like it to be, you can always re-test.
It’s best to test with new visitors rather than existing traffic so that your visitors don’t have any preset expectations of the site and you’re not interrupting the experience for existing site visitors.
Use your test results to find a balance between elements and content that converts, and what you want/need on the site. Sometimes what converts best is not always what you’d prefer to see on the site. If that happens, consider taking different elements and testing those against each other to find the right combination for all your business goals.

Stay tuned for deeper dive into some more specific forms of testing including the importance of headline testing and getting started with multivariate testing.