Why Multivariate Testing Matters for Marketers

Here at Metadata, you’ll hear us refer to multivariate testing a lot. Why do companies use multivariate testing, and how does it enhance your marketing processes and results?

To understand how you can get the most out of a marketing plan, you should know how multivariate testing works, and what it can do for your business.

Multivariate testing is essentially an approach that analyzes different variables to come up with the best solution for your audience or readership. It’s an improvement on some traditional methods that have started to help companies get on the right track and fine-tune their marketing processes, to promote better lead-to-opportunity conversion and pipeline acceleration.


The Old Way – A/B Testing

For marketers, a precursor to multivariate testing is A/B testing. It’s called A/B testing because you are testing two choices – A and B.

A/B testing works like this – you put both of these choices in front of an audience. In other words, you split your audience into two groups and show them one or the other of these options. Then you track what happens – you see how people engage with the content, where they move in the funnel. You then begin A/B testing that option versus another accordingly.


When A/B testing was new, data engineers were hyping it as an amazing way to take the guesswork out of marketing. Prior to the emergence of these new technologies and techniques, people really had to guess about what kind of marketing results were best – something any fan of AMC’s “Mad Men” would understand.


Multivariate Testing Approaches – Adding More Variables

Nowadays, companies have even better options. They don’t have to settle for a simple A/B comparison. That means instead of just analyzing two webpages or two channels or two fonts or headlines, marketers can analyze a spectrum of choices and figure out which one is best and most effective.

Multivariate testing multiplies and enlarges the testing experience. Where A/B testing only uses a binary model, multivariate testing expands into a larger number of statistical variables.


Why That’s Good – Doing More with Multivariate Testing

Suppose a company has three major ways to reach people – a Facebook page, a web site and a Twitter feed. There’s a core message that can be sent out over each of those three media.

Multivariate testing can probe this essential problem where A/B testing couldn’t. Companies can get hard results about which of the three channels can outperform the others.


But that’s not where it ends, either. Suppose there are three slight variations to this core message that can still go out over each of these three channels. That’s nine possibilities to deal with.

Good multivariate testing services can easily handle this problem. They will test all of those possible permutations and bring back actionable results that can bring tremendous returns on investment.



Adding Machine Learning and Artificial Intelligence

Even before the advent of machine learning and artificial intelligence techniques, multivariate testing was pretty great. But now you get an enhanced ability of machines to automate marketing processes and other decision-making tasks. You get multivariate testing on steroids.

A lot of what artificial intelligence is about is doing more with less human handling. Maybe in an original multivariate testing plan, the computer just came back with statistical results and showed them to people on a dashboard.

A demand generation platform that employs artificial intelligence or machine learning can take that data and make its own decisions. This will take even more of the labor and effort out of directing a marketing plan.

Think about this company we were talking about above. Instead of having marketing people take the raw numbers and analyze them, and manually refine the different permutations of offers and channels, your demand gen platform can actually make those decisions in close to real time and post the content themselves. You could essentially have self-running marketing programs that really don’t require much human intervention at all.

Your marketing team saves a ton of manual effort, and you can much more quickly hone in on the combinations of offers and channels that convert the best and drive pipeline and revenue.

When you really understand what multivariate testing can do, you’ll be excited about the ways that it can improve your business results. In the next post in this series, we’ll go into some of the examples of what today’s firms are doing with this cutting-edge marketing methodology.

Meanwhile, let us know if we can help you add multivariate testing to your demand gen efforts!

Multivariate Testing for Demand Generation Results

Why are B2B companies increasingly seeking to conduct multivariate ad testing? In our last blogs, we pointed out what the strategy is, and how it’s done. Here are some additional reasons why companies invest in multivariate testing, and why they get such great returns on their investments.

Sorting Out Channels

As a business, you have many options for attracting and communicating to your customers. In the old days, it was mainly the yellow pages, print coupons and word-of-mouth. Add radio and TV, and today’s social media platforms, like Facebook and Twitter, as well as email and other channels that utilize mobile devices. Simply put, you have to figure out how messages should stream across these diverse channels.

Multivariate testing accomplishes this well. It looks at every message and its makeup and layout, but it also analyzes competing channels. If you’re going to get more conversion from a Facebook post that you would from a tweet, multivariate testing can figure that out for you, so that you always place a message in the appropre channel.

 Crafting Your Message

Writers also know that the blank page is a chaotic place. There are infinite ways to write a communication to a customer. How long should a message be? What kind of tone should writers use?

 Solving Creativity Puzzles

Everybody’s talking about the user experience these days – marketing gurus and executives are recognizing that one of the best ways to compete as a business is to enhance your user experience. In demand generation, this means delivering your offers in a user-friendly and attractive format, one that draws them in and entices them to view ad respond to your message.

The trick, though, is that you never quite know which types of imagery will attract people, and which ones will confuse them or turn them off to the business. Online marketing is a tricky thing – in analyzing what’s sticky and what’s attractive to a target audience, there’s lots of nuance and detail. Multivariate testing helps marketers sort through the ‘noise’ and find the ‘signal,’ and that has immense value for nearly any kind of company.

Realize the Benefits of Multivariate Testing

Most businesses simply don’t have the in-house skills and resources to come up with a complex multivariate ad testing plan.

This isn’t done with a simple workstation or algorithm – it’s done with a sophisticated software platform that intelligently applies artificial intelligence and machine learning to conduct multivariate experiments at a velocity and scale that cannot be matched by in-house marketing staff or agencies.

The Metadata closed-loop demand gen platform and associated managed services enable B2B firms to realize the benefits of multivariate testing. We look at a particular client’s needs and build a program that will get you the actionable intelligence that drives your business forward. Think about using these services to dramatically improve your business in 2018, get more market share, improve lead conversion rates and fill the top of your sales funnel.

Want to learn more? Let us know if we can help you add multivariate testing to your demand gen efforts!

Examples of Multivariate Testing in Marketing

In a previous post, we talked about what multivariate testing is, and how it helps with selling. To review, this sophisticated marketing science gives you more insight into what’s going to work best to attract customers and grow your business. That’s a big help when you want to make sure that your company continues to grow sales pipeline and revenues.

Here are some examples of how multivariate testing can be applied to marketing.


New Audiences and the “Smart” Sales Funnel 

One situation where a good multivariate test can increase conversion and revenue is with a new audience segment.

Take a look at this article detailing a project by Jawbone, a maker of wearable devices for personal fitness.

In this case, we’re talking about a celebrity endorsement on the “Oprah’s Favorite Things” list. Now, Oprah Winfrey is a big name, and getting that attention from her is going to drive new people toward your products. So what does multivariate testing have to do with it?

Reading through the story, you can see that Jawbone’s marketers came at this from a particular angle of new audience education and personalized marketing content. By presenting many different options, and looking at specific metrics, the company was able to take better advantage of the opportunity in question.

Multivariate testing can help you to set up a sales funnel that’s more than just a simple path. When you get more involved marketing tools on board, you’re able to make better deals all the way through the process.

Creating the Perfect Ice Cream Sundae – or Marketing Email

The value of multivariate testing is about combinations.


Suppose a company is trying to design the perfect direct email campaign, and focusing in on one initial email with its content and layout. Let’s make it a company with a cutting-edge product – a smartphone app that helps with residential and commercial real estate. How do the business leaders get people to try it out?

Over at Mailjet, Bea Redondo Tejedor  points out how this is likely to go. Marketers can look at every aspect of the email, to try to tie them together in the best possible way. They’ll see which “flavors” go together and attract the customer’s attention. By contrast, they will also see which flavors don’t go together at all. A particular call to action may be jarring after a given headline. With good multivariate testing, marketers get to the bottom of how to optimize their content – and that adds enormous value to a campaign.


The Retailer’s Seasonal Schedule

Here’s another case study that shows the value of multivariate testing.

A retail shop in a small town (called Bob’s Family Store) sells all sorts of convenience items. The business also has an online store where e-commerce marketing is a major driver for the business.


In both of these aspects of the business, the business owners are looking at the calendar to figure out when best to target their audience in the community. They’re doing particular kinds of marketing on holidays and weekends. Before Black Friday, they’re busy for a week solid pushing a ‘buy local’ message to take advantage of that one big shopping day.

This is another situation where multivariate testing can make all the difference. This local retailer needs their messages to be optimized. They need to pack the biggest punch possible on certain days, and at certain times of the day.

This page from Marketingland goes into more details about using multivariate testing to work on an annual or seasonal schedule – pointing out that it might come down to something as simple as different “Buy Now” or “Sale” buttons online around holidays like Easter, Christmas, Halloween or Thanksgiving. The piece also points out that “reducing barriers to purchase” is one of the best ways to streamline conversion and boost the deal-making power of a business.


Selling Tech Services

Retailers are generally most interested in multivariate testing to reach their consumer audiences – but multivariate testing is rapidly moving into the B2B domain.


In this particular case study, a hosting company performed multivariate testing on it’s content to significantly improve lead conversion rates.

In a blog post, one of the business leaders reveals the key results in terms of conversion rates that run the gamut from 2.7% to an impressive 7.5%.

You can see how the company, Hawk Hosting, played with particular title content, bullet points and graphic design in order to get these hard results.

These case studies give you a window into how much small design changes affect customer mentality.

We know that marketing is a complex science – but now detailed multivariate testing can tell us a lot more about how that very complex science works.

The case studies above all have a common takeaway: multivariate testing can take a lot of manual effort and time to complete.


Multivariate Testing for Demand Generation

Unfortunately, many companies don’t have the in-house staff and the time to set up and run multivariate test campaigns, so they settle for relatively simple A/B testing. A/B testing can improve conversion rates, but it will take longer to see results.

Forward-thinking marketers are therefore taking advantage of next-generation demand generation solutions like metadata.io, that use machine learning and artificial intelligence to automate multivariate testing, and free up marketers to focus on higher-value activities.

Stay tuned for an upcoming post in which we’ll show some of the results that Metadata customers are generating using the Metadata AI Operator to optimize their demand generation efforts. 

Meanwhile, let us know if we can help you add multivariate testing to your demand gen efforts!