Digital Marketing Blog

A/B Testing

An Introduction to A/B Testing: Create Emails that Drive Revenue

Email marketing is still one of the most powerful ways to connect your business with a targeted audience. It is a fundamental tool in driving leads to your website, and ultimately, fueling business growth.

One of the best ways to help ensure your campaigns are delivering the maximum ROI is through A/B testing. A/B tests allow marketers to take on the role of an email scientist and experiment with different email variables to discover which type of email works best for your business. Here’s how you can get started.


Select Your Variables

The first step when diving into A/B testing is deciding what part of your email you think could be improved, and how you can change it to develop a better email. The variable you choose should be something that you believe will have a meaningful impact on how users engage with your email.

Also, be sure to only test one variable at a time. Testing multiple variables at once will not give you accurate results. So keep it simple and take it one test at a time.

Here are a few common email variables you could try to change for your first A/B test:

  • Your email’s subject line (Example: “Register Now!” vs. “Join Us for a Free Seminar”)
  • Call to action (Perhaps a button works better than a text link)
  • Preview text
  • Body copy
  • Design (Studies show that different colors invoke different actions. How could you change certain colors to influence user behavior?)
  • Layout (Where is your call to action? How is your text positioned?)
  • Length of email
  • Images
  • Your offer (“Buy one, get one deal” vs. “15% off”)

When selecting a variable to test, make sure it’s one that you believe will have a statistically significant impact on your email campaigns. Don’t waste your time testing variables that are not going to drive meaningful actions from users. For example, running an A/B test on the footer of your email would not make sense because the footer has no meaningful measure of driving user engagement. Be sure to test something that will be able to be tracked, and help accomplish a strategic objective.


Who to Test

Now that you have the variable you wish to test selected, you need to decide who your test audience should be. When selecting who will receive your test, you must always remember that you will need enough recipients to gain meaningful insights from your test. The best way to gather meaningful data is to test your A/B emails with all of your contacts. However, there are a few exceptions to this best practice:

  • Financial Limitations. Depending on your budget, and the method in which your email service provider charges for distributions, it may be a strategic decision to reduce the size of your test list, especially when being charged per email address.
  • Big Changes. If the variable you are testing is a radical change for your emails, it may be better to test the change with a smaller group in case the outcome is not successful. By testing a more drastic change on a smaller group, you can recover and ensure the remainder of your list receives the best quality email between the two variables.
  • Timed Offers. If you are focused on getting the highest possible return on an offer you will not be sending out again, or one that has a limited time frame, you can send the test to a portion of your contacts. Then, determine which variable was more successful, and have that email sent to the remainder of your contacts, therefore maximizing the success of your limited time offer.

When dividing any list, always be sure it is done completely randomly to avoid skewing your test results.

The overall rule of thumb when doing A/B tests is that larger sample sizes yield more accurate results. You want genuine, honest user data, even if it means that your proposed subject line won’t yield more opens for your email. It’s all about refining your emails, trying new things and having the data to back up your future decisions.


Define Success

When designing an A/B test, always have your end goal in mind. What is it that you’re seeking to accomplish? If your new subject line is designed to increase open rates, then measure the success of this test by the change in open rate. Not in clicks or conversions. Always keep your goal in mind. The variables you test need to be designed to accomplish your goal.


Testing Tools

If you use an email campaign software, odds are it has a built-in A/B testing tool. HubSpot, MailChimp, Campaign Monitor and Active Campaign all have a built-in A/B testing tool that comes with your subscription. These tools take a lot of the manual work out of A/B testing, and excel at displaying the test results in easy-to-read formats. If your email program does not have built-in A/B testing tools, you can always perform the tests manually. This will require more work on the front end when designing the emails and splitting the lists, as well as taking the time to compare the test results.


The Results

Now that you’ve delivered some great test emails to your contact list, it’s time to see what’s working and what isn’t. Then again, is it time? Be sure you don’t analyze your results too early. Allow several days for people to open your email and interact with it.

Now that it’s really time to measure your results, here are three key statistics you will want to pay attention to:

  • Open rate
  • Click-through rate
  • Conversion rate

The open rate and click-through rate of your emails should always be analyzed when email marketing. These key stats help you understand where your messaging is working and where it isn’t. They are simple metrics to try to improve through A/B testing, and are important in helping drive website conversions.

But why track conversion rate? Say your end goal is for users to download an e-book from your company website. Wouldn’t conversion rate be tied to the effectiveness of your landing page rather than your email marketing efforts? Not exactly.

Your landing page plays a large part in converting your website visitors. After all, that is its sole purpose. But your email marketing strategy plays a large part in conversion rate too.

If you see a 10 percent increase in visitors to your landing page, you should also see a corresponding increase in your landing page conversion rate. If this isn’t happening, something might be amiss with your email communications.

If you are seeing an increase in click-through rate but no increase in conversions, this likely signifies a disconnect between what your email marketing and your landing page is communicating. These results tell a story of users who clicked on a link in your email expecting one thing, and then lost interest as a result of being offered something different.

Remember, for inbound marketers, the end goal of your marketing campaigns is to drive leads for your business and fuel revenue growth. If your click-through users are not converting, take a step back and look at your email’s offer compared to your landing page’s offer. Do they line up? It’s important to make sure that the messaging delivered in your email is consistent with that of the landing page. A disconnect between these two messages will lose new business for your company.



A/B testing is a terrific way to improve the impact of your email marketing efforts. It is not something that should be done once, and then halted for an indefinite period of time. You should always be testing! Try new ideas, craft new messages, think creatively and continue to rediscover what your users want and will respond to in your emails. Doing so will drive more leads to your website, and help turn your email marketing campaign into a revenue-driving machine.

A/B testing provides an exciting chance for your business to continually make consumer-driven improvements. I’d love to hear how your business is planning to grow through your email marketing strategy, or answer any questions you may have. Email me at

New Call-to-action

This entry was posted in Digital Marketing, Mailchimp RSS, Strategic Marketing and tagged , , , , , , , , , . Bookmark the permalink. Follow any comments here with the RSS feed for this post. Comments are closed, but you can leave a trackback: Trackback URL.
© Copyright 2019 Skoda Minotti | Privacy Policy | Disclaimer | Remote Support
Cleveland 440-449-6800 | Akron 330-668-1100 | Tampa 813-288-8826
Website designed and developed by Skoda Minotti Strategic Marketing