Testing times

When we ask email marketing folks what testing they do when sending email campaigns, the list is usually pretty short. In fact, it’s often limited to subject lines and list segmentation.

Whilst segmenting your list is probably the number one type of testing you should be doing, there are many other tests you can do to get a solid understanding of how well your email marketing is doing and where there is room to improve.

Start by clearly defining what you want to achieve and set benchmarks so you can keep track of where you are along the way. Typical KPIs are opens, clicks, purchases and revenue. You should also define who you want to test various elements to and the list sample size.

Understand the resource required to run your tests. Testing adds to the amount of work for each campaign and those involving multiple creative treatments and list segments will be more time consuming than subject line or timing tests. A campaign that takes three days to set up and deploy may end up costing you more than the profit it generates.

So here are a few things you can test to help incrementally improve your email marketing response rates and ultimately your bottom line.

List segmentation
List testing is the number one thing to test because if you’re not reaching the right audience with the right message they will tune out and your open and click rates will decline. The more you understand your data, the better selections you can make so mine your historical information, create clusters of people and assign them categories.

Template designs
The effectiveness of your templates, from responsiveness on different devices to length, imagery, copy, offers and call-to-action should be tested as some will perform better than others. Select a few variations to split test until you narrow down a set of templates that give the best results, then fine tune design, layout and content from there. Take a look at Email design: How to improve customer engagement


Preview pane view
Many email systems show a truncated version of emails in a preview pane so test to ensure that the main message you want to get across is contained within this area.

What images should you use and how many should your include in your emails? Avoid stock photos where possible, particularly generic ones. People need to be excited by what they see. All images should be pin sharp – enough of us have high resolution screens on our laptops, tablets and smartphones which make crystal clear images look amazing but show up poor quality ones. Test an email with just a few key images against one with lots and see how click rates compare, but remember that too much choice may mean people not choosing anything.

Make the most of  images too. Our attention spans are shortening, more so on a ‘phone where we have to look closer and are likely surrounded by distraction so you need to get peoples attention fast: Seconds out – how do people read email these days?. Try a mix of photos and graphics and test placement to see where people click most.

Many webmail operators, Gmail in particular, block images by default so design and code templates so people still know what your email is about and encouraged to click ‘show content’.

Text content
How much copy, what should I say and what’s my tone of voice? Be concise and avoid flowery language, people want to know what your email is about and they want to know fast. If they can’t work it out quickly your email will likely be deleted. Ensure your template is coded to be responsive so the copy adapts and can be read easily on a small screen. Test a conversational tone against a more professional one to see which approach people prefer.

Content personalisation
Dynamic content, data-driven content, one-to-one content – whatever you call it you should try and make the email as relevant to your audience as you can. No-one likes having their time wasted reading emails that don’t relate to their interests. By segmenting your list and changing the content of your emails for each group you can test the uplift you get against a control group with a generic template.

Test CTA’s in button and link form. For buttons, try colour, style, wording and placement. Be more inventive than ‘click here’, test words like ‘Learn more’, ‘Take a look’, even ‘Go!’. Don’t forget images as calls-to action – people expect images to take you to what’s in the picture so make these inspirational. Review the results of where the clicks are highest and compare images, buttons and text links. Check out Calls-to-action. What are they and how can I use them effectively?

Offer type
Test a range of offers with a focus on different list segments. Certain groups, such as those based on purchase history, personal life stage, or your top email engagers may respond differently.

Subject line
Seen as the easiest of all tests as all you need to do is come up with two or more subject lines and run a basic A/B split test on control lists to see which gets the best open rate. As with all tests, log all results so you can track what works and what doesn’t. For more on subject lines, see Open Sesame: Making your subject line more magical


How many emails you send to your list will depend on a range of factors but, as a rule, we are more open to a higher frequency of emails than we were, say, five years ago as we interact with emails differently now, see Upping email frequency – people forget very quickly these days. When increasing the number of emails you send, pay close attention to changes in open and unsubscribe rates.

Email timing
Reports abound of the best day and time to send emails. For B2C emails, this isn’t as a it used to be as we can all open emails whenever and wherever we like – and we do. All of the timeline graphs we see show a classic steep and immediate curve of opens straight after sending followed by a similar curve back down and then bubbling along for a few days with small humps in the morning and evening. B2B campaigns lean more towards sending during office hours (whatever they are these days!) with morning being the preferred time. However, it’s worth testing to see if any uplift can be gained, particularly for any time-sensitive events. Create a matrix as you test and plot your KPI’s so you have a record of any differences in performance, bearing in mind that your results can be affected by external factors such as major sporting events, holidays and hot Summer days.

Personal vs. alias email address
People won’t usually see the sending email address until they open your email, but who it’s from can have an impact of how personal the email feels. There are still many companies using ‘noreply@‘ which sends the message, ‘we want you to buy our stuff but don’t want to talk to you’. An alias such as ‘shop@‘ or even ‘info@‘ is better but a real name is likely better still for a more human touch.

Name personalisation in the subject line
Not one of my favourites as it has been around for a while, is overused and takes up valuable character space. It’s also commonly seen in those emails none of us wants – those from third party list rental companies sending emails on behalf of products and services we have haven’t asked for and have little interest in. However, It is eye catching so if you’re a trusted brand or organisation then it’s worth testing, especially as it’s quick and easy to do.

Email from field
My view is that your company name or a recognisable brand should be in the from field – people want to know who the email is from. You can test this though by changing it to a persons name (or anything you like really) to see what people prefer.

Plain Text vs. HTML
Best practice dictates sending a text version of an email to accompany the HTML version, even though most people won’t ever see it. However, there is evidence to support some people preferring text emails and, as this is another easy test to do, it’s worth seeing if plain text emails get you an increase in clicks.

Social sharing
Rather than just adding social media icons in your email, test how people respond to your activity by creating more of a story around your social presence and give them reasons to like or follow your company.

Social proof
What our peers say about brands, companies and their experiences has a high trust factor. Split test your creatives so one version includes Tweets, reviews or a Facebook post vouching for an offer, product or service as they can have high click rates and generate a strong feel-good factor.


Landing pages
To continue the great email experience, ensure what people see after clicking or tapping a link is what they would expect and user friendly. Many of the same elements above can be tested such as layout, imagery and calls-to-action. Find out what it takes to make the user journey as easy as possible to get people to do the thing you want them to do. Quick Tip Guide: 5 Basic Tips to a good landing page

Continue the process
Testing should never stop. Over time you will create a knowledge bank of what works for your audience and build up inventories of offer types, creative treatments, subject lines and techniques which will become a powerful arsenal for your future email campaigns. Much of your work going forward will involve minor tweaks and fine tuning but we all know the internet never stands still so be prepared for ‘the next big thing’ and how you can adapt your email marketing to maximise your advantage.