The fastest way to fill a low-incidence study with fraud is a yes/no screener. "Do you own a boat?" Everyone who wants the incentive says yes.

A screener has three jobs: find the right people, manage quotas, and protect the study. Most only do the first.

Key takeaways

  • Never let respondents guess the target from the question wording.
  • Use multi-select lists with plausible distractors.
  • Add one or two traps, not ten.
  • Re-check key facts later in the survey and against profile data.

Hide the target

Instead of "Do you manage cloud software purchases?", ask "Which of these do you personally approve purchases for?" and list eight categories, one of which is cloud software. The qualifier is buried in normal-looking options.

Same with brands. Show a list of ten and ask which they've bought in the past six months. Don't ask about your client's brand alone.

Red herrings and traps

A fake brand or a product that doesn't exist in the market catches people who select everything. Keep it believable; an obviously silly option only catches the laziest bots. And use one or two traps. Stack too many and real respondents start to feel tested, which hurts completion.

Consistency checks

Ask a key fact twice in different ways. Company size in the screener, number of employees in the main survey. Age in the screener, year of birth later. Real people answer consistently. Fraudsters often don't remember what they claimed.

If your sample provider has profile data, compare against it. A respondent who was a nurse last month and a CFO today deserves a second look.

Mistakes that bias the sample

Overly strict screeners cut incidence and push field toward whoever is left, often heavy survey takers. Leading wording ("As a busy professional...") nudges people to qualify. And long screeners on mobile quietly drop younger respondents.

For B2B work especially, pair the screener with role verification. We go through that process in verifying IT decision makers.

FAQ

How long should a survey screener be?

Aim for under two minutes, usually 5 to 10 questions. Anything longer raises dropout, especially on mobile, and screen-outs start costing you goodwill.

Do red herring questions annoy real respondents?

One or two well-designed ones don't. They look like normal answer options. Problems start when traps are obvious or repeated, which makes honest respondents feel accused.