Emails – Why and how should you A/B test them?

During our analysis and research phase of conversion optimisation projects, we not only look at on-site conversion, but also consider whether email flows offer opportunities to increase a business’ conversion rate.

For example, for one of our Lead Generation clients, Scrap Car Comparison, we recently ran an email A/B test which increased their collections by 25%.

In this post we discuss how emails can be improved, the measures to look at and how these measures indicate what to change.

Why are emails important?

Emails can often be overlooked – your team may not regularly see them, unless they are in the habit of spot checking them regularly. But they are an important form of communication with your customers at various points in their journey with your brand – for example:

  • Welcome flows
  • Order confirmations (often one of the most important yet overlooked emails)
  • Basket abandonments
  • Product review requests
  • Back in stock notifications
  • Nurture flows
  • Cross selling other relevant products
  • Follow-up emails – for instance, your next vaccination dose is due in x weeks or reorder reminders for frequently purchased products that users may not necessarily be subscribed to.

So where do we start?

Firstly we create a visual board of all our client’s email flows – Miro is a good tool to use for this. This can also help identify opportunities for new flows.

Secondly, we wish to understand the volume of each email type – as we want to make the most impact, and higher volumes mean A/B tests will reach significance sooner.

Next we make sure all emails are being tagged correctly – for instance, UTM parameters are set correctly to make sure the traffic is all allocated correctly.

Then we look at the Click Through Rate (CTR) and the Open Rate for each email. This helps to indicate which ones could be improved and what type of improvements are needed.

What does a Low Click Through Rate (CTR) on high volume emails indicate?

If we see a low CTR, then the first questions we consider are:

  • Is messaging a problem?
  • Are the users being given a reason to click?
  • Can we use urgency or scarcity to get people to start to take action?
  • Is the text clear, appropriately sized and does it render gracefully on mobiles?
  • Do the users have a clear Call to Action (CTA)?
  • Are the users being given a reason to come back to your site, or have you given it all away in the email?
  • Does it feel like a human is addressing their concerns on an individual level – for example, using their name and conversational language? Or does it feel like the email was generated by a system? This can be a big turn off.

What does a Low Open Rate indicate?

If users aren’t opening the emails the most likely areas to investigate are:

  • Is the subject line intriguing?
  • How compelling is the teaser content?
  • Does the email feel personalised or tailored to their specific interests?

How do we fix these issues?

Once we have identified which emails show the most potential for improvement and understood what is causing users not to interact with them, our expert consultants and designers work together to craft a carefully worded and designed email, with clear CTAs and appropriate teaser content.

These new emails are then tested against the existing emails, as an A/B test, allowing the impact of the change to be measured, before winners are rolled out to 100% of users.

If you’d like to talk to us more about how we could run similar A/B testing for your emails and website in general, please get in touch and one of our consultants will be happy to discuss this with you.

To find out what
we could do for you:

Get in touch
Get in touch