multivariate testing

What is multivariate testing and what does it involve? If you plan to use your analytical creativity to drive a metrics driven marketing strategy, you are going to need to know the answer to this question.

Multivariate testing is all about optimization. It's all about implementing a structured testing process to determine the optimal way to achieve an objective. When considered in the context of Online Marketing, multivariate testing can be applied to any number of your marketing tactics, however, Google Adwords and Landing page creative are the most popular tactics for multivariate testing at the moment.

Taking Google adwords as an example, multivariate testing can be implemented to determine optimal ad creative. The process involves creating multiple 'recipes' of ad creative and testing their performance against a control group.

Not surprisingly, there are already a number of organizations that have proprietary multivariate testing software that you can invest in and implement. Offermatica and Memetrics are two leading providers in this area. I was fortunate to be see a presentation from Offermatica's Jonathan Mendez in Chicago last year. He really opened my eyes to the benefit of multivariate testing, particularly in the Adwords space where I was investing a lot of budget at the time (50K + per month).

Unfortunately, well for marketing geeks like me anyway, most New Zealand websites do not really generate high enough numbers (of any sort) to justify investment in multivariate testing software. This is not to say that you need high numbers to do basic multivariate testing. Every New Zealand website that plans to be profitable should at least master the art of the simplest form of multivariate testing, the A/B split test.

If you already have a Google account and are actively investing in adwords, you should definitely check out Google Optimizer, a beta landing page multivariate testing tool from Google. It's free and you can always rely on Google to provide excellent training and help. A great starting point for those that are new to multivariate testing.

If you are interested in progressing a bit faster, or have already got some experience in basic A/B split testing you may be interested in this list of Key Terms for data Driven Marketers. A list of definitions for the golden words of the analytically creative marketer.....

The future of marketing is upon us, and (Sorry Kevin) its not about lovemarks- it's all about facts and numbers.

Comments

[Billy Shih]

I love the opening (2nd sentence). I've always thought about multivariate optimization that way but never put together a sentence so eloquently. Thanks!

[Alasdair Bailey]

Hey James,

Good post, great to read that the word is spreading 'down under' on site optimisation!

You missed a provider from your list; Maxymiser is the leading UK and European provider of A/B, multivariate and behavioural targeting technologies. Your readers in our neck of the woods might be interested to hear they have a local provider.

Kind regards,
Alasdair

(I work for this company)

[James Hurst]

Nice blog, I'm glad you see the value in MVT. As a sales rep for Memetrics, we typically say that a potential optimization customer needs to have more than 1000 outcomes or conversions a month and spend more than $1 million a year in online marketing for an investment in a top tier multivariate testing solution to make business sense. Or more simply, you can say, 'what is a 10% lift worth to your business?'

cheers