Multivariate Or A/B? Which is Most Effective for Testing

In the 19th-century, merchant John Wanamaker, famously said, “half of my advertising is wasted; I just don’t know which half.” 

Now, in 2019, that couldn’t be further from the truth. Tools and strategies for measuring the performance of digital marketing efforts have evolved and advanced in the last few decades. Marketers can now measure and analyse their campaigns at an extremely granular level. The most successful marketers assess the performance of their campaigns regularly to eliminate what is not working, and optimise what is, in order to achieve the best return on investment (ROI). 

You need to learn what works for your business and do more of it to ensure you are performing to your full potential. In order to do this, testing is essential. There are two main types of testing that digital marketers can implement in their campaigns: Multivariate and A/B testing. The following discusses the benefits of each.

A/B Testing

A/B testing, also known as ‘split testing’, is a method of website optimisation where conversion rates of two different versions of a page (version A and version B) are compared using the traffic that comes to the site. With an A/B test, visitors are split equally between the pages, and marketers track how visitors interact with the pages that are shown. This can be used to track the videos that they watch, the buttons they click, or whether they sign up for an email newsletter. By assessing traffic in this way, marketers can determine which version of the page is most effective in reaching their goals.

A/B testing is a relatively simple way of evaluating the design of a page and the elements on it. It is most often used when a marketer wants to test two different designs to find out which is more effective. For example, one landing page may have a navigation bar with links to other pages on the site, while the other may be a standalone page. After enough visitors have been to both pages, the number of conversions can be compared to identify which is more effective. 

The main advantage of A/B testing is that it is a relatively easy, yet powerful testing method. As the number of tracked variables is small, these tests can provide reliable data quickly. They also don’t require a huge amount of traffic to run. A/B testing can also be a good introduction to testing for a team that has not previously done it as it is relatively simple to implement and generates results fast. 

Multivariate Testing

Multivariate testing uses the same principle as A/B testing, but compares more variables, and provides more information about how the variables interact with each other. The same as with an A/B test, the traffic to a website is split between two different versions. The multivariate test will then measure the effective that each design and combination of elements on the page has on the ultimate goal and the visitors’ interactions with the site.

Multivariate testing is most commonly used on pages where several elements need to be tested, for example, a landing page that includes a sign-up form, header text, call to action buttons and a footer. Rather than having just two combinations of design like in an A/B test, to run a multivariate test, marketers may create two sign-up forms with a different number of fields, four versions of headline text, and two footers. Visitors are then funnelled to all of the possible combinations of these elements.  Once the test has finished, the elements on each of the page variations are compared to each other to identify the page that is the best performing, as well as the elements that can take the most credit for the performance. 

As there are more variations that need to be tested, requiring more traffic, it takes longer to obtain enough data to carry out effective multivariate testing. However, if your site has enough traffic to run these tests, multivariate testing is an effective way to help redesign your landing pages with the elements that will have the most impact. Multivariate tests are particularly useful for campaigns that need landing pages, such as for Google Ads or social media advertising campaigns. The data gathered can ensure conversion rates are as high as possible, and learnings from the test can be applied to future campaigns.

Get in touch with the digital marketing experts at Digital Squad for more information, or with assistance in optimising your digital campaigns. 

If you’re ready to take the next step to improve your digital strategy, talk to the experts at Digital squad, your digital marketing agency Auckland. Whether that be social media marketing Auckland, Google Shopping Auckland and Bing ads, we are here to help you out. Contact us and find out how you can improve your conversions.