New research from Ai2 tested how reliably large language models can evaluate the novelty of scientific research ideas. The result: changing just a few words in the prompt shifted the model’s accuracy in identifying which of two ideas is more novel by more than 50 percentage points. The study examined six models: GPT-5.1, GPT-5.2, GPT-5.4, Claude Sonnet 4.5, Claude Opus 4.5, and Claude Opus 4.6.
What the research found
Researchers tested whether LLMs can consistently judge which research idea is more novel. They found that seemingly minor changes in how the evaluation was defined produced dramatically different judgments on the same ideas. Retrieval and adding additional reasoning did not consistently improve judgment. Two dedicated systems for evaluating novelty produced worse results.
The core finding: same model, same ideas, small change in prompt – completely different outcome.
Why it matters beyond the headline
Many organisations are considering investing in AI systems that evaluate, filter, or rank content: research proposals, customer requests, business decisions. The research shows that LLM judgment is not stable enough for critical decisions without human oversight.
This is not a problem that will be solved in the next generation of models. It is an architectural limitation: a language model generates probabilities, not objective judgment. A change in phrasing changes the statistical context, and therefore the output. If you are building a system that relies on LLM judgment, you are effectively building a system whose output depends on prompt phrasing more than on the content itself.
What to do this week
If you are already running a system that uses an LLM for evaluation or filtering, add human review on a random sample of 10% of outputs. If you are planning such a system, design human validation, periodic audits, and safety mechanisms from the start. Do not hand critical decisions – such as rejecting a proposal, filtering a lead, or approving a request – to an automated system without oversight.
If you need to evaluate or rank content automatically, consider building AI agents for business with built-in review mechanisms rather than a closed judgment system.
Sources
- Can we trust LLMs to judge whether a research idea is novel? · Hugging Face
Frequently asked questions
Can prompt engineering improve LLM judgment stability?
The research shows that retrieval and adding additional reasoning did not consistently improve judgment. A small prompt change can shift output by 50%+, meaning prompt engineering does not solve the underlying stability problem.
Are newer models more stable in judgment?
The study tested six different models, including advanced versions of GPT and Claude, and all showed similar instability. This is an architectural limitation of language models, not a training flaw that will be fixed in the next version.