Nobody wants an AI that is 94% right. Everybody wants one that knows which 6% to hand over.

A model that is 94% accurate and always answers will be wrong about your customers six times in every hundred, at random, with total confidence, forever.

A model that is 88% accurate, abstains on the 15% it is least sure about, and routes those to a human, produces a system that is wrong about 1% of the time and hands you a clean queue of hard cases.

The second model has worse accuracy and is enormously more valuable. Everything below follows from that.

Calibration beats accuracy

The property you want is calibration: when the system says 0.9, it should be right about 90% of the time. When it says 0.6, about 60%. A well-calibrated mediocre model is deployable, because you can draw a line under it and know what falls below. A poorly-calibrated excellent model is not, because its confidence tells you nothing about when to intervene.

This is measurable and almost nobody measures it. Take a few hundred cases with known answers, bucket the system's outputs by stated confidence, and plot actual accuracy per bucket. If the line is roughly diagonal, you can set thresholds. If it is flat — 95% confidence on everything, right 80% of the time — you have no gate to build, and the honest conclusion is that this system cannot be safely automated at any level yet.

Abstention has to be a real outcome

"I don't know" must be a first-class output of the system, not a fallback when generation fails. In practice that means it has explicit triggers, each of which is a policy decision rather than a model one:

• Retrieval was thin. Nothing relevant found, or only weakly relevant. Do not let the model paper over a search failure with fluent prose — that is the single most common way hallucinations reach a customer.
• Sources disagree. The help centre says 30 days, the contract says 60. That is not an ambiguity to resolve, it is a human decision, and often a bug in your documentation that you now get to find.
• The stakes exceed the remit. Contract language, legal threats, regulated advice, anything discretionary. Confidence is irrelevant here — blast radius decides, not certainty.
• The customer is angry. Detected frustration, escalating language, or repeat contact on the same issue. A correct answer delivered to someone who has now written four times is still a failure.
• Novelty. The case looks unlike anything in the grounding corpus. Distribution shift is the failure mode that arrives without warning — month-end, a product launch, an outage — and novelty detection is how you notice before your customers do.

The number to run the system on

Track two things together and neither one alone:

Coverage — the share of cases the system handled without a human.
Precision on covered cases — how often it was right on the ones it took.

You want to move coverage up while holding precision at an agreed floor. That reframes the whole programme in a way everyone in the building can agree on: nobody is arguing about whether the AI is "good", they are watching two numbers with a contract between them. And it makes the trade-off explicit, which is what lets an operations director sign off.

A system at 40% coverage and 99.5% precision is a good system, and it is a much better business than one at 95% coverage and 94% precision. The first one is quietly removing 40% of the work. The second one is generating a customer complaint every twenty tickets and it will be switched off by Christmas.

What this means for how you buy

When a vendor leads with an accuracy figure, the useful follow-ups are:

1. What is your abstention rate, and what triggers it?
2. Show me the calibration curve on data like mine.
3. What happens to an abstained case — where does it go, and how fast?
4. Can I set the threshold, per action type, myself?
5. When you are wrong, how do I find out, and how do I undo it?

Question five is the one that separates a product from a demo. Everything else is negotiable.

Systems that know their limits get deployed. Systems that are merely accurate get piloted, admired, and switched off in month three.