Evaluating LLMs before they reach production
2 July 2026 · 5 min read · Nintech AI
Every LLM feature starts with a demo that works. The model answers the hard question, the room nods, the project is funded. Then it meets real users, and the failure modes nobody rehearsed — ambiguous phrasing, missing context, adversarial inputs — arrive on day one. The gap between demo and dependable is not prompt cleverness. It is evaluation.
Why demos lie
A demo samples the distribution of questions its presenter knows the system handles. Production samples the distribution of questions your users actually have — including the 15% that are underspecified, the 5% that are about something else entirely, and the handful that are actively hostile. A system that scores perfectly in the meeting can fail a third of real traffic without anyone noticing, because LLM failures are fluent. Wrong answers arrive with the same confident tone as right ones.
Build the eval set from real tickets
The best evaluation data already exists in your business: support tickets, dispatcher emails, contract review notes, search logs. Take a few hundred real cases, have domain experts write the acceptable answers, and record what makes an answer unacceptable. This becomes your eval set — not a synthetic benchmark, but a compressed copy of your actual problem. On a recent logistics engagement, 2,400 historical dispatcher tickets became the harness that every model change had to pass; it caught regressions that no reviewer would have spotted by sampling outputs.
Gate changes in CI, like tests
Once the eval set exists, wire it into the same pipeline that runs your unit tests. A model swap, a prompt edit, a retrieval change — each produces a scorecard against the eval set before it can merge. The cultural effect is bigger than the technical one: model updates stop being events and become boring, reviewable changes. Teams that ship eleven model updates in six months without incident are not lucky; they are gated.
Fine-tune when evals say so
Fine-tuning is a tool, not a milestone. If prompted baselines pass your evals, ship them and bank the simplicity. Fine-tune when the eval set shows a persistent gap that prompting cannot close — domain vocabulary, output structure, tone under edge cases — and measure the tuned model against the same harness. The eval set, not the technique, is the asset. It outlives every model you will ever deploy on top of it.
Working on something like this? Talk to an engineer.