Human review is the gold standard for judging open-ended answers, but it does not scale to every pull request. That is why so many teams now use a second model as a judge: give it the input, the output and a rubric, and ask for a score.
It works surprisingly well — with a few caveats that are easy to miss.
Where a model judge shines
- Checking criteria that are clear to a person but hard to code, like tone, completeness or whether an answer actually addresses the question.
- Comparing two versions side by side, where relative judgements are more reliable than absolute scores.
- Triaging large volumes of production traffic so humans only review the borderline cases.
The known failure modes
Model judges have biases of their own. They can prefer longer answers, favour whichever option is shown first, reward confident wording over correct content, and rate outputs from similar models more kindly. None of these are fatal, but all of them are invisible unless you look.
Calibrate before you rely on it
The fix is the same as for any measuring instrument: calibrate it. Have people label a sample of outputs using the same rubric, run the judge on the same sample, and measure how often they agree. Where they disagree, read the cases — usually the rubric is ambiguous, not the judge.
Rubric: faithfulness (1–3)
3 = every claim is supported by the provided context
2 = minor unsupported detail, core answer supported
1 = contains a claim that contradicts or is absent from the context
Return: {"score": <1-3>, "reason": "<one sentence>"}A practical checklist
- Keep rubrics short and concrete, with an example for each score.
- Ask for a reason alongside the score, and read a sample of reasons every week.
- Randomise the order in pairwise comparisons.
- Re-check agreement with humans whenever you change the judge model or the rubric.
Treated this way, a model judge is one strong signal among several, not the final word — which is exactly what you want from an evaluation system.