When in doubt, test!

When in doubt, test!

The word “test” scares many adults. Well, at least it scares me. It brings to mind school exams and report card day, with my teacher standing in front of the classroom and calling out names in order of best to worst score for the trimester, which takes a while for 70 students (yes, public shaming was all the rage at my primary school).

But what about tests in the email marketing context? Very few of us in the industry think of it, let alone do it. Yet, we should because quite frankly, all the other cool kids are doing it and enjoying all the perks.

So, what exactly do I mean by testing? And why should you even bother with it?

Basic Test

This is as easy as it gets. You create your message, then send it to yourself and/or others. Your goal here is to make sure that everything looks good. You’re interested in knowing whether:

  • Text and images are properly displayed or formatted
  • The personalisation in your message actually works
  • Links in the message work correctly

A basic test of your email serves as a sanity check, allowing you to correct any embarrassing errors before they get into the hands of your recipients.

A/B Test

This one can be as easy or as complex as you want. The key thing here is that it requires some planning on your part. A basic A/B test goes something like this:

  • Create your email
  • Setup various versions of your email. For example, you could have the same email with different Subject lines. Or the same email with two different blocks of text. It’s up to you. Just plan for it
  • Decide which segment will be receiving the email
  • Send each version of your email to a sample of your segment (e.g. 5% receives version A and another 5% receives version B)
  • Monitor the performance indicator(s) that matter to you (e.g. open rate, click through rate, etc.) and decide which version you want to send to the remaining members of your segment

While this may seem like a lot of work, remember that there are a few key benefits to investing in A/B testing:

  • No guess work which subject line or image will appeal the most to recipients. An A/B test allows your segment members to speak for themselves, and for you to give them what they want.
  • You can decide which indicators will be used to determine success of the test. If you’re very active on social networks you may want to base your decision on the rate at which your content is shared on Facebook. Or maybe you’ll base your decision on number of clicks to your landing page.


Multivariate Test

With multivariate testing, the world is your oyster. Whereas A/B tests can cumbersome and difficult to manage when you want to test combinations of elements, multivariate testing allows you to automate and test every permutation and combination of elements that your heart desires. For example, which combination of style sheet, subject line, and header image will results in high open rates? Simply define your test variables and the tool will generate emails that cover all possible combinations.

Multivariate testing allows you to not only automate the testing but the decision process as well. With the right tool, you can define how long the test should run before a “winning” version is chosen. In other cases, you can do it all in real time. Meaning that the person who opens an email before a winning version is chosen could see different content if they open the email after selection of the winning version.

The flexibility that comes with multivariate testing means you can truly follow one of the golden rules of email marketing: target, target, target. It’s not enough to know that you want to send your email to a single, Peurto Rican mothers, under age 30, living in the New York area. Your goal is to ensure that everything about the email – from content to layout – appeals to, and engages, the segment.

This form of testing is a much more precise and, dare I say, scientific, approach to testing. The more you do this sort of testing, the more you get to know your customers and in the long it results in less frustration about how to design your emails and what content to provide. With more targeted emails you naturally improve conversion and overall return on investment

04 Oct 2010

Share this