AI Engineering
Evaluation and observability for AI in production
Language model systems drift silently. Here is how to build the measurement that catches it.
By Whitesoft · · 6 min read
Conventional software fails loudly. A language model system fails quietly: answers get slightly worse, a provider updates a model version, a new document type arrives, costs creep up. Without deliberate measurement, the first sign is a complaint from a customer or an auditor.
Build the evaluation set first
Collect real inputs with expected outputs, reviewed by the people who know the domain. Start with fifty to a few hundred examples covering the common cases and the edge cases that matter. This set is the most valuable artefact in the project. It is used to choose the model, to test every change and to detect drift.
Measure four things continuously
- Quality: accuracy or acceptance against the evaluation set, and sampled human review of live traffic
- Cost: tokens and dollars per task, tracked against the business case
- Latency: end-to-end response time at the percentiles users feel
- Behaviour: refusal rates, escalations to humans, tool call failures and retrieval misses
Trace everything
Every request should be traceable through retrieval, prompt, model call and any tools invoked, with inputs and outputs logged in line with your privacy obligations. When quality drops, you need to see whether retrieval returned the wrong document, the prompt changed, or the model behaved differently.
Set a review rhythm
Weekly review of quality and cost dashboards during the first months, then monthly. Quarterly, re-run the evaluation set against alternative models. Treat a provider model update as a release that requires re-testing, because it is one.