I've looked at a lot of raw data files over the years, and bad responses tend to fall into a few families. Each one has a different cause, and treating them all the same way (usually "delete anything suspicious") removes real people along with the fraud.

Bottom line

  • Speeders, straight-liners and bots are different problems with different fixes.
  • Single-rule removal is a blunt tool. Combine flags and score them.
  • Some bad data is caused by the questionnaire, not the respondent.
  • Keep a removal log. Clients should be able to see what was taken out and why.

Speeders

A speeder finishes far faster than anyone could read the questions. The common rule is "under one-third of median LOI," and it's a reasonable start. But set it per section, not for the whole survey. Someone who skips a long optional block legitimately will look fast overall.

Also compare devices. Mobile respondents often move faster through single-choice questions and slower through grids. One threshold for both will over-flag one group.

Straight-liners

Straight-lining is picking the same answer down a whole grid. Sometimes it's fatigue, sometimes it's fraud, and sometimes the respondent genuinely feels the same about all ten statements. Before you blame people, look at the grid. Twelve rows of agree/disagree on page seven of a 25-minute survey invites exactly this behavior.

Useful flags: zero variance across a grid, alternating patterns, and contradictions with a reversed statement.

Bots and scripted respondents

Bots are where things got harder. Older scripts were easy to spot: identical timings, gibberish open ends, the same device over and over. Newer automation uses real browsers, randomized timing and language models for text. CAPTCHA alone doesn't stop it.

What still works is layering: device and network signals at entry, behavior during the survey (pointer movement, paste events, focus changes), and content checks on open ends. That's the structure of QualityShield, and it's why we check at the door rather than only cleaning afterward. We go deeper on this in AI agents taking surveys.

Removing bad data without adding bias

Score each respondent on several flags and remove on the combination. Someone who is a bit fast but writes a thoughtful open end is probably fine. Someone who is fast, straight-lines and pastes a generic paragraph is not.

Then document it. A removal log with counts by reason is standard practice for us, and it's worth asking any sample provider for one.

FAQ

What percentage of survey responses are fraudulent?

It varies widely by source, audience and incentive. Clean, verified panels see low single digits after entry checks. Open traffic exchanges without strong screening can be far worse. Ask your provider for their removal rate by project.

Should speeders always be removed?

Not automatically. Remove respondents who are extremely fast on content they had to read, especially when another flag agrees. A single borderline timing flag on its own is weak evidence.