Getting Started with Multivariate Testing for Marketing
by Reena Leone
We’ve walked you through the basics of A/B testing and stressed its importance when it comes to maximizing conversion on your website. While A/B testing usually focuses on testing two or more variations of a single element against a Control, you can test more than two elements of a website at the same time to find the right combination to hit your goals; whether that’s engagement, revenue, etc. This is called multivariate testing.
What is multivariate testing?
In the simplest terms, multivariate testing tests website variations that have more than one page element changed. For example, instead of testing just changing a headline in Variation 1 and Variation 2 and then comparing the results, you’d change the headline and the feature image. Multivariate testing is not limited to just two elements at a time; you can test as many elements as you want to create many different combinations of those elements. This lets you essentially, tweak your website experience to get the best result possible.
What to test
When conducting your multivariate test, just like with A/B testing, you need to consider what your goals are in order to select the right elements to test. Are you looking for more engagement around an article? Or maybe you want to to increase your email sign ups? Looking at the latter, consider testing the following:
- Button placement
- Button color
- Email sign up text
For example, testing button color and text might look like this:
Pros and cons of multivariate testing
One of the advantages of multivariate testing compared to standard A/B testing is being able to fine tune your website. As we mentioned before, sometimes seemingly small changes can have an impact.
Multivariate testing lets you refine your website experience as much as you want to, however, before you line up a bunch of elements to test, there are a few things to keep in mind.
The only limit here is time; however, we all know time is money, as the saying goes. The more elements you add in, the more variations you create, the longer it will take to get conclusive results. Remember, during multivariate testing, your audience will be divided evenly by the number of varied experiences you create and if you don’t have a ton of new visitor traffic daily, it might take a while to have enough data to determine which version of the website is converting the best.
To cut down on too many different variations, try pairing elements that go together in a test like headline text and images, or button color and placement. Or, instead of testing several different elements, try testing 2-3 but with different a few different variations within those elements. For example, if you’re testing an image and a headline, instead of having multiple images and text, consider testing a couple different image options with maybe 2-4 headline options. Multivariate testing is only as complex as you make it.
If you’re looking to go more in depth with your testing, whether it’s A/B or multivariate, or even want gain more insight on analytics, a good resource to check out is Occam’s Razor by Avinash Kaushik, an expert on all things analytics and Google’s own Digital Marketing Evangelist.