In my last blog I wrote about the importance of careful survey design for data quality and respectful engagement with respondents. In this blog I share best practices of survey design and how they can be easily implemented in your research projects.
How often do you hear “we need a final survey” before scripting. I don’t believe there is a final version when you begin scripting. We actually need three final surveys to get to our master final version that will be ready before sending out to participants.
Stage 1: When setting up our questionnaire we start writing our research into questions. In other words that’s your blueprint and stage 1 of your survey. Your final questionnaire ready to script.
Stage 2: But now comes the fun part, we need to rewrite some of the content into a more user friendly experience for the respondent. You don’t just want to fire computerised questions to people, otherwise they will just tick off the answers. We need to make sure people read our questions and take time before answering. We want them to think instead of just filling in some answers.
Stage 3: Everything that you have done in stage 1 and 2 needs to be tested.To start with you don’t know how people will respond to your designed survey, so it is always good to have a small group of people (preferable your target participants) fill in your designed survey. Let them give you feedback on the design and content. This will also allow you to check their behaviour in the data. Do you need to rephrase your questions to improve the quality of open text answers? Do you see enough answer variance in each of the statements of your grid questions? How quickly did they fill in the survey? How much time did they spend per question? With the feedback and all your checks you are able to adjust your survey to a better version.
Probably you will say, although this is the ideal process, it just takes too much time and increases costs. Yes, that’s correct. But what if you run a bad survey and worse if people fill in that survey? Bad designed surveys lead to bad data, which can lead to wrong results which again can lead to incorrect business decisions. I would prefer to take some extra time and costs to have a well designed survey rather than saving a little bit on the design process.
Some tips & tricks to think differently about your survey content.
It is not only to make it all just visually more attractive. It is also about how you write your questions.
Make it more human and personal. When something is personal people feel more engaged. Set the right stage and tone of voice. What is your role in the survey? How do you want to talk to your participants? Are you the researcher, friend, colleague or expert?
We need to motivate people to take the time to read and think before answering your questions. Make it interesting and challenge people!
Shorten questions and answer text. Just ask only what you really need and write questions like you speak. Write simple and easy to understand, leaving out unnecessary scene settings.
Self-explaining questions don’t need extra instructions. Avoid repeating question text in answer text and vice versa. There is no need to explain the scale in the question text if you show the scale also in the answers. Or instructions like, click on next to continue, only one answer possible, describe in your own words, etc. It sounds like a small thing but imagine you have this in every question it is just an overload of information that is unnecessary.
It is a good thing to make surveys not too long. But the length of the survey is, most of the time, not the real problem why people don’t complete a survey. It has much more to do with repetition and the boring way of asking questions. It is much better to improve your survey flow, making it a pleasant journey.
Group and combine questions. Add multiple questions about the same topic on one page. This reduces next button clicks and avoids multiple pages. It is quicker to fill in and it provides a clean view.
Instead of repeating a set of questions you can combine these.
For example, these three questions:
Can easily combined into 1 question:
Using images and icons are very powerful and will be processed more quickly in the brain. Especially when using repeating questions it is helpful to recognise the answer quickly.
Add extra screens. There is a big chance people are not willing to read a big block of text. Making it simple and clean you can be more certain that people really read and have noticed what the purpose of the topic is. Don’t be afraid to divide long intro’s or long blocks of text into separate multiple pages. Try to shorten text and visualise. More pages doesn’t mean the survey becomes longer. It improves the survey flow and will only reduce the time needed to complete the survey as well as keeping the participant focussed.
Often there is no introduction before a certain topic starts. We jump directly into the big battery of grid statements. An extra screen helps to make the survey fluent.
Autofill questions. If you have already available panel information, show this to the respondent with the option to change if it is not correct. Don’t ask the same questions each time if that information is already available. It is one of the biggest frustrations from panel members.
Autocomplete. Speed up the open answers by already using a predefined answer list. By typing certain characters some matching responses are shown.
Autonext where possible. Add autonext options to save a lot of clicks. Add in a back button if you think people could have answered too quickly.
Provide instant feedback. Everybody wants to know that it matters what they do. Engage the participant by providing them instant feedback during the survey. For example: feedback about their answers, i.e. what kind of shopper they are, how other people responded, facts, fun, etc. Reward people by the time and effort they take to answer your questions rather than just getting an incentive depended on the length of the interview.
Add pauses People can’t absorb information beyond a certain rate. Adding pauses allow people to stay focused. Even when people don’t take a break it is good they know they have just finished a certain part and have the option to breathe again.
In my next blog I will share best practices about designing single and multi-response questions.