How to get the most out of employee surveysJust because you can physically do something doesn’t mean you can do it well. Similarly, sending out a survey question doesn’t mean it’s a good, well-constructed survey question, one that will prompt useful feedback which can be acted on. These 8 steps will get you infinitely better results.
1. Don’t frame questions that give unactionable answers
People make this mistake all the time with survey questions. For example, having a ‘strongly agree’ to ‘strongly disagree’ scale to a statement such as, “the company newsletter should be published once a month”. If you get ‘strongly agree’, that’s fine, but if you get ‘somewhat disagree’ or ‘strongly disagree’ you don’t know if the newsletter should be published more or less frequently.
Similarly having a ‘agree’ ‘disagree’ scale for ‘I receive the right amount of information from the company’, if you get a ‘disagree’ answer you won’t know if they are getting too much information or too little. So, ask questions that lead to solutions. Be sure to use a scale for the question that actually lets you know if you are doing the right thing or not.
2. Don’t ask two different things in the same question
Again, this happens a lot and leads to answers that mislead. For example, ‘communications from top management are open and honest’, on a ‘strongly agree’ to ‘strongly disagree’ scale. In this instance there is nothing wrong with the scale, but there is a problem with including ‘open’ and ‘honest’ because they are two very different things. Companies can be very open with their employees but also dishonest, and other companies can be honest but not at all open, with management being inaccessible and closed.
A second problem with this question is lack of definition. ‘Top management’ might mean different things to different employees. On the shopfloor it could mean their immediate boss and those above them, and at another level it could mean the chief executive and her/his team. So, don’t put too many things in one question and be sure to define terms that could be misinterpreted.
3. Avoid having ‘yes’ or ‘no’ type questions on a four or five point scale (strongly agree, somewhat agree, somewhat disagree, strongly disagree)
Internal communications and employee survey expert Angela Sinickas has seen survey questions that gave these scale options to ‘my manager conducts team meetings every week’. Framing this in a ‘agree’ ‘disagree’ scale is ridiculous as it’s a yes or no situation: the managers either conduct weekly meetings or they don’t. Similarly, a four point ‘strongly agree’ ‘strongly disagree’ scale is inappropriate for this statement: ‘our company’s reputation is non-existent. Again, ‘non-existent’ is a yes or no kind of answer.
If the statement was posed as ‘our company’s reputation is excellent’ and ‘our company’s reputation is good’, the latter gives the greatest gradation of response, the best range, as it allows people to scale their answer up or down. They can ‘strongly agree that it is good, or they can ‘strongly disagree’ that it’s good (in which case you can reasonably take it that they think your reputation is bad). The ‘excellent’ option doesn’t give the same clarity, because people who think it’s only good could choose either ‘somewhat agree’ or ‘somewhat disagree’ so it’s best to use a word that is medium positive.
4. Don’t build in assumptions – and avoid words like ‘always’ and ‘never’
‘I enjoy working here because I am always kept informed of things that affect me in my job’. There is so much wrong with this. First, there’s the assumption that the person enjoys working there and so it’s a very difficult question to answer. If you want to know if they are happy working for the company, put it in a separate question.
Avoid ‘always’ and ‘never’ because usually people can think of at least one example of where it’s not true, so they won’t choose the strongly worded option. Instead, use ‘almost always’ or ‘almost never’.
5. Ask only those who would have an informed opinion
If you are trying to find out how useful something was to somebody, then first find out if the person answering the question actually used it, be it a publication, a booth at a trade show, or whatever. Otherwise you will get answers that distort your feedback.
To avoid this, simply ask a ‘yes’ ‘no’ question at the outset: did you read X publication; have you visited our booth at a trade show in the past 12 months?. If they answer ‘no’ you move on to another question, but if they say ‘yes’ follow it with a four point scale option to ‘the trade show booth provided information that was valuable to me’. So, make sure only people who have a legitimate opinion are being asked to evaluate it.
6. Phrasing is everything: keep the questions clear, simple and easily understood
Here are three examples of statements around the same issue:
a) Not getting an answer to my questions is all too ubiquitous
b) Not infrequently, I fail to get answers to my questions
c) It is easy to get my questions answered
The problem with “a)” is the use of ‘ubiquitous’, a word rarely used in normal conversation, and might not be understood by everybody. It’s important to pitch your survey at a reading level that matches your audience’s reading level. All too often, people use big words to appear intelligent or important, but Mark Twain nailed it: Don’t use a five-dollar word when a fifty-cent one will do. The issue with “b)” is that it’s a double negative, which is something to avoid at all costs as it’s confusing.
7. Structuring response scales
How many response options should the survey questions/ statements give?
If it’s a yes or no question then it’s two or three (yes, no, don’t know). Otherwise it’s usually between four and six. Best not to have too many options. 4 point scale: strongly agree, somewhat agree, somewhat disagree, strongly disagree. 6 point scale: excellent, very good, good, fair, poor, awful.
Don’t forget neutral, average and not applicable options
Neutral would be including a ‘no opinion’ option whereas average would be the ‘just right’ option in the following example 5 point scale: The amount of emailed information I receive from the company is: far too little, slightly too little, just right, slightly too much, far too much.
The non-applicable is also very important in a question such as this: In the past 12 months communications from the company has: become much better, become a little better, stayed the same, become a little worse, become much worse, have not been here 12 months.
If the latter option wasn’t included for someone who might have joined the company only a couple of months previously it would lead to distorted data.
8. How to administer the survey
Does everybody have to be surveyed?
It’s not necessary to survey everybody: you can do random samples and still be able to project the results to the whole population as long as you make sure you sample them truly randomly, that you are not over-sampling some groups and undersampling others.
How many to randomly sample?
If your survey group is 2,000 or more (it could go up to 20,000, 200,000 or 2 million) once you get a response rate of around 400* it will be statistically valid and reflective of the group as a whole once the selection is random.
*A response rate of 400 has a margin of error of 4% for 2,000 people surveyed, and 5% for 20,000 and 200,000. A 5% margin of error is built into most political polls and is accepted as statistically valid.
With Poppulo you can check the pulse of your organization by embedding surveys and quick polls into your emails, enabling two-way communication and giving employees a voice.