Research
Rewording a prompt shifted LLM judgment by over 50 points
An Ai2 study found a few reworded prompt words moved LLM accuracy by over 50 percentage points. Why that matters for lead scoring and automated decisions.
What it means for youIf a language model scores your leads, sorts customer requests or rates content, don't assume the output is stable just because it looks reasonable. Build a small set of cases a person has already decided, run each through several wordings of the same prompt, and see whether the answers move. Any decision that shifts with phrasing should stay with a human, and a prompt edit should be treated as a change to business logic.