Open ends used to be the easiest place to catch fraud. Gibberish, copied product descriptions, "good good good." Not anymore. Now the bad answers are fluent, polite and three sentences long. They just don't sound like anyone in particular.

Key takeaways

  • AI-written open ends are common enough that every study should check for them.
  • Look at how the text arrived (paste, typing rhythm) as well as what it says.
  • Compare answers across respondents. Generated text clusters.
  • Ask personal, specific questions that a model can't answer convincingly.

What generated answers look like

They're balanced to a fault. "There are several factors that influence my decision, including price, quality and convenience." Grammatically perfect, no typos, no slang, no specifics. Real people write "honestly just whatever's on sale at Target" or "the blue one because my kid likes it."

Detection signals

How it arrived. A 60-word answer that appears in one paste event after two seconds on the page is a strong flag. Real typing has pauses, corrections and a rhythm.

How it compares. Run semantic similarity across all open ends in the study. Twenty respondents who phrase the same idea in nearly the same structure, from different "people," is a pattern you'd never see naturally.

How it fits. Does the open end match the closed answers? Someone who rated the product 2 out of 10 but writes a glowing paragraph is either confused or not human.

These checks run inside the content layer of QualityShield alongside device and behavior signals.

Questions that resist AI answers

  • Ask about personal experience: "Tell us about the last time you..." rather than "What do you think about...".
  • Keep it short and concrete: "What store did you buy it from, and why that one?"
  • Occasionally ask for something only the respondent knows, like what they had for breakfast before a food diary task.

Handle flags fairly

Some respondents use spell-check or translation tools, especially outside English-speaking markets. That's not fraud. Score AI likelihood as one input, combine it with other flags, and have a human review borderline cases before removal. Throwing out every polished answer will bias against careful writers.

What this means for qual

If you're coding open ends into themes, generated text inflates the "safe" middle themes and hides the edges. Clean first, then code. We talk about hybrid designs in AI-moderated qualitative research.

FAQ

Can you detect ChatGPT answers in surveys?

Not with a single test, but combining paste detection, typing behavior, cross-respondent similarity and consistency with closed-ended answers catches most of them.

Should AI-assisted answers be removed?

Fully generated answers from someone who didn't engage should be removed. Light assistance like spell-check or translation usually shouldn't. Review borderline cases rather than applying one blanket rule.