Identifying Bad Data in Customer Surveys
All of us ask questions. If we’re lucky, other people have answers. Be wary though, because it’s highly likely that not all of them are correct! Now, there’s nothing inherently bad about receiving a ‘wrong answer’ – it’s up to you whether or not you will take someone else’s reply as fact after all – but sometimes misinformation can lead to disastrous results. This is especially true for those who conduct customer surveys as the basis for their whole study relies on their gathered information being accurate. Thus, identifying fraudulent entries in their data is a chief concern.
Understandably it can be quite difficult to weed out the truth from the lies, especially in customer surveys which usually amount to a wide range and volume of people. But there are a few telltale signs that most surveyors know how to look out for such as:
Speed
There exist respondents who willfully agree to take your survey and yet for whatever reason (such as being busy or your survey is too long or boring), refuse to read through it thoroughly and provide thoughtful, meaningful answers. These are commonly referred to as the speeders, and these are the people who take an impossibly quick time to answer all the given questions. There are countermeasures against this type of tactic – for manual surveys, a manual timer can be employed so they do not have the option to speed through the survey. As for the online variety, there are also advanced online survey tools that are able to track time and tell you how long a respondent took to complete your survey. If they fall within the average time, then chances are their data is good.
Patterns
Be it a conscious decision or otherwise, some respondents choose to employ a pattern when haphazardly answering a multiple choice survey. The most common patterns are the straight line (wherein only on column or row of answers is selected) or the zigzag (where the answers alternate one after the other). Like the speeders, this behavior indicates that the answers do not in any way depict the respondent’s opinion, and should thus be scrapped. This method is most often employed in incentivized surveys.
Inconsistency
While some may exert a bit more effort into ensuring their answers don’t seem like a total farce, sometimes their inattention can still be detected. How? A strategy often applied by surveyors is to slyly ask the same question twice. This is done by asking about a certain subject once, and then repurposing the question so that the same answer should be expected when the subject comes up again. When applying this strategy try not to make it too obvious; refrain from completely copy-pasting the same question twice. That would be shoddy questioning on your part.
Another way to gauge the respondent’s engagement is through the use of trap questions. Ask simple, obvious things like “is the sun brighter than the moon?” If the answer is wrong, then it is highly likely that the respondent is not answering properly. As for essay type or question-and-answer surveys, one word answers or broken sentences are a clear indicator of inattention and should be flagged for inconsistency.
Are you seeing any of the following patterns in your results? Then maybe it’s time to flag it as bad data and discard it! Although not all surveyors employ data cleaning methods similar to those seen above, it’s imperative that you do when you can. It may be time-consuming and labor intensive, but it sure beats investing in misinformation.