AI Engineering
Right model, right use case, right economics
Frontier models, small models, classical machine learning or no model at all: how we decide.
By Whitesoft · · 7 min read
A new model is released, the benchmarks are impressive, and within a week someone asks whether you should be using it. The honest answer is almost always 'for what?'. Model selection is an engineering decision with commercial consequences, and the newest or largest option is rarely the right one for a specific task.
The dimensions that matter
- Accuracy on your task, measured on your data, not on public benchmarks
- Cost per task at your expected volume, including context size and retries
- Latency the workflow can tolerate
- Data residency, privacy and contractual constraints on where data may be processed
- Explainability and auditability requirements
- Operational maturity: rate limits, uptime, versioning and deprecation behaviour
A rough map
| Task shape | Often the right tool |
|---|---|
| Open-ended drafting, complex reasoning, low volume | Frontier language model |
| Well-scoped extraction or classification, high volume | Small language model or fine-tuned classifier |
| Forecasting, anomaly detection, optimisation on structured data | Classical machine learning |
| Answering from a defined document set | Retrieval-augmented generation with a mid-sized model |
| Multi-step work across systems with judgement | Agent with deterministic tools and human escalation |
| Rules that are known and stable | Ordinary software. No model. |
Evaluate before you commit
Build an evaluation set from real examples with expected outputs before comparing models. Run each candidate against it and record accuracy, cost and latency. The result is frequently that a model a fraction of the size and price meets the quality bar for the task. Where it does not, you will know exactly why the larger model earns its cost.
Design to switch
Put model access behind a gateway, keep prompts and evaluation sets under version control, and avoid provider-specific features unless they are worth the lock-in. Prices and capabilities will change. The organisations that benefit are the ones that can re-test and move.