Poppulo
Measurement

See the difference in your comms results with A/B testing

By 

 — June 9th, 2018

See the difference in your comms results with A/B testing

As the creator of a 5-Step Marketing Process, I always need to stay on top of the latest and greatest forms of communication.

Despite being in a highly connected world, it can still be tough to decide how to best communicate my clients’ brands to their customers. One method I use to help with these decisions is A/B testing.

A/B testing is the practice of using two slightly different versions of the same media at the same time to determine which one has a higher conversion rate.

The conversion rate measured (clicks, survey submissions, etc.) depends entirely on the media being used and your communication goals.

You may have previously thought that A/B testing is only used in marketing, but it is a highly effective tool for any form of communication, internal or external.

There is an endless list of comparisons you might want to consider including in you’re A /B testing including:

  • Images: Are your employees more likely to complete a survey that has a green background or a blue background?
  • Tone: Do your employees enjoy the casual tone you use in your newsletters, or would they be more likely to read to the end if they were written more professionally?
  • Subject Lines: Which action verbs best persuade your employees to open your emails? Are they more drawn to short subject lines or ones with more detail?
  • Surveys: What is the perfect way to phrase a question in order to receive more detailed answers? How many questions are too many, causing employees to open surveys but choose not to complete them?
  • Training Documents: Do new recruits learn the ropes faster with information presented in a table or as bullet points? Do they retain more information if it’s presented orally or in a corporate video?
  • Professional Opportunities: Which guest speaker will increase attendance at your next Lunch and Learn? Does mentioning “Free refreshments will be provided” really increase RSVPs to a workshop?

In short, A/B testing can be applied to many of your communication needs and will ultimately increase your employee engagement.

Having used A/B testing to conduct market research for many of my clients, I have a couple of tips you can use before diving in:

Start Small

The purpose of A/B testing is to gauge the effectiveness of the communication before delivering it to your entire staff. Send the two versions to separate small groups, make any necessary changes based on the results, and then send the winning communication to everyone.

This way, if the initial experiment doesn’t produce the desired results, you can try again with different groups.

Don’t Overreach

When performing A/B testing on a particular communication, I suggest changing no more than one specific item at a time between version A and version B.

If you do, it could be difficult to pinpoint exactly which aspect of the communication resonated with your staff.

Consider Timing

Is your business entering its busiest time of the year? Are multiple employees escaping poor weather to go on vacation? Is it flu season?

The answers to these questions can drastically affect your data. Unless you want to constantly change your sample group, consider what might be going on in the lives of your employees.

Not everyone opens every email they receive every day. I suggest running your tests for at least a couple weeks so that you have the opportunity to collect enough information.

Keep Testing

Once you experience significant results from an A/B test, keep in mind that just because version A beat version B (or vice versa), that doesn’t necessarily mean it is the ultimate winner.

Your next workshop registration or survey participation could be even better. Replace the least engaging of the versions with something new and test again, or you can start an entirely new test for another form of communication.

Steadily watch your engagement grow with each new test.

Remember What Matters

You want your communications to be as close to perfect as possible, but do you need to test if your click-through rate improves based on the use of a single comma?

Although A/B testing can help you with many aspects of communication, it’s important to stay focused on the data that really matters to you. I encourage to always be testing something, but that something needs to be worth your time.

Open Your Mind

You may be hesitant to bother with A/B testing if you already have your heart set on one version in particular. But testing is still important because you never know what you might learn.

You could find out that your initial assumption was correct, or the test could reveal that your employees have a strong leaning towards the other version.

Or you could discover that there wasn’t enough engagement with either one, so more options need to be tested. Without proof, you can never be certain what will work.

Use a Tracking System

Imagine investing precious company resources into designing two versions of a communication and scheduling the ideal time to send them to a carefully chosen sample group only to not learn anything.

If you plan to use A/B testing, you need a way to track employee engagement. For email communications, I cannot emphasize enough that you can’t simply go through any email provider.

Most don’t offer tracking, or at least not at the level you would most benefit from. You need a full email communications program that offers detailed but user-friendly tracking information.

By implementing A/B testing, you can monitor results monthly, weekly, or even daily to regularly improve your internal communications.

I encourage you to take the time to learn how to set up an A/B test or hire a professional agency to help you discover how it may benefit your specific communication needs.

The best on communications delivered weekly to your inbox.
Future-Proof Your Leadership Comms: Trends and Strategies for 2025
UPCOMING WEBINAR – DECEMBER 11TH

Future-Proof Your Leadership Comms: Trends and Strategies for 2025

View more