Poppulo

IC Matters

Measurement

See the difference in your comms results with A/B testing

By 

 — June 9th, 2018

See the difference in your comms results with A/B testing

As the creator of a 5-Step Marketing Process, I always need to stay on top of the latest and greatest forms of communication.

Despite being in a highly connected world, it can still be tough to decide how to best communicate my clients’ brands to their customers. One method I use to help with these decisions is A/B testing.

A/B testing is the practice of using two slightly different versions of the same media at the same time to determine which one has a higher conversion rate.

The conversion rate measured (clicks, survey submissions, etc.) depends entirely on the media being used and your communication goals.

You may have previously thought that A/B testing is only used in marketing, but it is a highly effective tool for any form of communication, internal or external.

There is an endless list of comparisons you might want to consider including in you’re A /B testing including:

  • Images: Are your employees more likely to complete a survey that has a green background or a blue background?
  • Tone: Do your employees enjoy the casual tone you use in your newsletters, or would they be more likely to read to the end if they were written more professionally?
  • Subject Lines: Which action verbs best persuade your employees to open your emails? Are they more drawn to short subject lines or ones with more detail?
  • Surveys: What is the perfect way to phrase a question in order to receive more detailed answers? How many questions are too many, causing employees to open surveys but choose not to complete them?
  • Training Documents: Do new recruits learn the ropes faster with information presented in a table or as bullet points? Do they retain more information if it’s presented orally or in a corporate video?
  • Professional Opportunities: Which guest speaker will increase attendance at your next Lunch and Learn? Does mentioning “Free refreshments will be provided” really increase RSVPs to a workshop?

In short, A/B testing can be applied to many of your communication needs and will ultimately increase your employee engagement.

Having used A/B testing to conduct market research for many of my clients, I have a couple of tips you can use before diving in:

Start Small

The purpose of A/B testing is to gauge the effectiveness of the communication before delivering it to your entire staff. Send the two versions to separate small groups, make any necessary changes based on the results, and then send the winning communication to everyone.

This way, if the initial experiment doesn’t produce the desired results, you can try again with different groups.

Don’t Overreach

When performing A/B testing on a particular communication, I suggest changing no more than one specific item at a time between version A and version B.

If you do, it could be difficult to pinpoint exactly which aspect of the communication resonated with your staff.

Consider Timing

Is your business entering its busiest time of the year? Are multiple employees escaping poor weather to go on vacation? Is it flu season?

The answers to these questions can drastically affect your data. Unless you want to constantly change your sample group, consider what might be going on in the lives of your employees.

Not everyone opens every email they receive every day. I suggest running your tests for at least a couple weeks so that you have the opportunity to collect enough information.

Keep Testing

Once you experience significant results from an A/B test, keep in mind that just because version A beat version B (or vice versa), that doesn’t necessarily mean it is the ultimate winner.

Your next workshop registration or survey participation could be even better. Replace the least engaging of the versions with something new and test again, or you can start an entirely new test for another form of communication.

Steadily watch your engagement grow with each new test.

Remember What Matters

You want your communications to be as close to perfect as possible, but do you need to test if your click-through rate improves based on the use of a single comma?

Although A/B testing can help you with many aspects of communication, it’s important to stay focused on the data that really matters to you. I encourage to always be testing something, but that something needs to be worth your time.

Open Your Mind

You may be hesitant to bother with A/B testing if you already have your heart set on one version in particular. But testing is still important because you never know what you might learn.

You could find out that your initial assumption was correct, or the test could reveal that your employees have a strong leaning towards the other version.

Or you could discover that there wasn’t enough engagement with either one, so more options need to be tested. Without proof, you can never be certain what will work.

Use a Tracking System

Imagine investing precious company resources into designing two versions of a communication and scheduling the ideal time to send them to a carefully chosen sample group only to not learn anything.

If you plan to use A/B testing, you need a way to track employee engagement. For email communications, I cannot emphasize enough that you can’t simply go through any email provider.

Most don’t offer tracking, or at least not at the level you would most benefit from. You need a full email communications program that offers detailed but user-friendly tracking information.

By implementing A/B testing, you can monitor results monthly, weekly, or even daily to regularly improve your internal communications.

I encourage you to take the time to learn how to set up an A/B test or hire a professional agency to help you discover how it may benefit your specific communication needs.

The best on employee communications delivered weekly to your inbox.

By clicking “Accept all cookies” you agree to the storing of cookies on your device to enhance your browsing experience, analyze site traffic, and serve tailored content and advertisements.

Cookies preferences

When you visit any website, it may store or retrieve information on your browser, mostly in the form of cookies. This information might be about you, your preferences or your device and is mostly used to make the site work as you expect it to. The information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies. Click on the different category headings to find out more and change our default settings. However, blocking some types of cookies may impact your experience of the site and the services we are able to offer.

Manage consent preferences

Strictly Necessary

Always Active

These cookies are necessary for our website to function. They do not store any personally identifiable information and are usually only set in response to actions made by you, such as setting your privacy preferences, logging in or filling in forms. You can set your browser to block these cookies, but some parts of the site will not work.

Functionality

Functionality cookies are used to remember your preferences. They make the site easier for you to navigate by remembering settings you have applied, detect if you’ve already seen a pop-up or auto-fill forms to make them easier for you to complete.

Targeting

Targeting cookies are used to deliver ads more relevant to you and your interests. These cookies can also be used to measure ad performance and provide recommendations.