Data Cleaning for surveys

This report is generated on-the-fly and cannot be retrieved after you leave this webpage.

Summary

anomalies were found. Of these are notably suspicious and should be removed.

These are examples where a respondent has:

• answered nearly randomly
• answered differently to other respondents on most questions
• broken away from the pattern of answers we see elsewhere in the data

Please note that the algorithms cannot distinguish between fraudulent and honest responses with 100% certainty, so not all the outliers definitely represent 'bad' data. The algorithms can only detect unlikely/odd/weird/strange cases, those that reside outside the wide space where the majority of respondents are found. Though fraudulent, unengaged respondents or 'bots' may be identified, anomalies can also be genuine. For example, 'a pensioner who commutes on a bike and owns a PlayStation 5'. Such a case, although unlikely, might be your very real but eccentric aunt.

Regardless, most analysts tend to remove as many anomalies as possible before attempting to build explanatory models. Retaining these responses, even if authentic, turns up the noise in the data, and drowns out the signal. That said, overzealous cleaning harms the representation of your data.

As such it is advised that you do not follow the algorithms blindly, particularly if many outliers are flagged. Case by case consideration is always advisable.

Suspicion Scores

To assist your due diligence, each respondent has received a score to reflect a level of suspicion.

To assign these scores the algorithms have considered your data in multidimensional space or the relationship amongst all the answers in your questionnaire. The plot below shows only 3 of these dimensions (20 to 30 questions required before an algorithm can suspecting anomalies). i.e. This plot is not intended to be insightful, rather it offers an intuitive, simplified depiction of how the algorithms viewed your dataset.

What makes each respondent suspicious?

It is important to appreciate that, by definition, each respondent will be anomalous for different reasons. The following plots are courtesy of Lunberg and Lee (2017) , and are designed to evidence why each respondent is considered to be anomalous. The bottom of a waterfall plot starts as the expected value, and then each row shows how each answer leads us to suspect the respondent is anomalous (orange) or real (blue). You can see how the combinations of these certain answers in your survey begins to raise suspicion.