Confidence, Abstention, and Escalation

Confidence, Abstention, and Escalation is the set of moves available when the system should not commit. A product that can decline, decline for a reason, and hand the work to somebody who can finish it is strictly better than one that answers everything, and building that takes more than reading a number the model volunteered.

Key takeaways

• A confidence score the model states in its own output is a token sequence produced because you asked for one. Nothing in training bound it to observed frequency, so treat calibration as a measurement you have to make.
• The usable signals sit outside the model: verification against a system of record, agreement across independent samples, and whether the retrieval step found any support at all.
• Abstention is a product state with a contract: what the user sees, what the system does not do, and which named queue receives the work.
• The threshold is a business decision, because it trades the cost of a wrong committed answer against the cost of an unnecessary decline, and those two costs differ by interaction shape.
• Over-abstention is a real failure. A feature that declines often enough to be unreliable gets routed around, and then you have the cost with none of the benefit.

Read this after Chapter 4, which built the commit point that a threshold check attaches to, and before Chapter 6, which routes work between models rather than to people. Measuring whether any of these signals is calibrated is the subject of the evaluation argument, and this chapter assumes that work is happening.

The extraction service returns a confidence field. It has done so since the second week, because somebody added confidence to the output schema and the model has been filling it in ever since.

The threshold is 0.85. Nobody remembers choosing 0.85.

Somebody finally plots accepted results against that field. The curve is almost flat: things scored 0.95 are wrong about as often as things scored 0.7. The gate has been passing everything, and it has been doing so with the reassuring appearance of a control.

Self-reported confidence is a number, not a measurement

The failure here is not that the model lies about its certainty. It is that the number was never connected to anything.

When you ask for a confidence field, the model produces a plausible confidence field. It is generating text conditioned on a schema, the same way it generates the rest of the response. There is no separate calibrated estimator being consulted. The number is output, not introspection.

Even where a genuine probability is available, such as token log-probabilities from the serving layer, the mapping to correctness is not free. Guo, Pleiss, Sun and Weinberger showed at ICML in 2017 that modern deep networks are poorly calibrated in one direction: the average confidence of a modern architecture sat substantially above its accuracy, where an older and smaller network's average confidence had tracked its accuracy closely. Confidence drifted upward as networks improved.

That is a result about image classifiers. Quoting it as though it settles the language model case would be dishonest. What it establishes is the shape of the problem: confidence and correctness are separate quantities that have to be related empirically.

The relation is a curve you can draw. Bucket a few thousand outputs by their score, measure observed accuracy per bucket, and plot one against the other. If the buckets separate, the score is a signal. If they do not, you have a decoration with a decimal point.

Abstention is a product state with a contract

Given that, the design question moves from scoring to what happens on a low score, and that needs to be a first-class state rather than an error path.

Abstention is the system deliberately declining to produce a committed answer, while still doing something useful with the request. It has three parts, and a design that omits any of them is not abstention. What the user is told. What the system explicitly does not do. And where the work goes next.

That third part is the one teams skip. A decline with no destination is a dead end, and users learn within a week that the feature stops working on hard inputs.

Abstention is also not a hedge. A hedge is a committed answer wrapped in qualifying language, which puts the burden of judgement on the reader while still asserting the content. Hedging is worse than either alternative. It looks like caution and functions as a claim.

The signals worth thresholding come from outside the model

Once you stop asking the model how sure it is, several better questions become available, and they share a property: something other than the generator produces the answer.

Verification against a system of record is the strongest. Does the identifier resolve. Do the line items sum to the total. Does the named customer exist, in this tenant, under this contract. These are boolean and cheap, and Chapter 2 called this the verifiability gate for exactly this reason.

Agreement across independent samples is the second. Run the same input twice, or through two different models, and compare the fields that matter. Disagreement is a real signal about the input's difficulty, and it costs extra calls, which is a trade Chapter 7 prices.

Retrieval coverage is the third, and it is badly underused. Where the answer is meant to be grounded in a corpus, ask whether retrieval returned a passage that supports the claim. No support found is a far more honest reason to decline than a low score.

Input shape is the fourth and the crudest. A document forty times longer than anything in your evaluation set, in a language you did not test, in a format you have not seen: those are cheap tells that you are outside the range where your measured error rate applies.

The threshold is a business decision, not a model parameter

Any of those signals gives you a dial. Where the dial sits is not a machine learning question, because it trades two costs that only the product knows.

The bottom row is where the arithmetic changes character. For a reversible action, an unnecessary decline and a wrong answer are both ordinary operating costs, and you can tune between them on volume. For an irreversible one, the two costs are not the same kind of thing at all, so the threshold is set by the worst case rather than by the average.

Read the table as a warning about defaults. A single global threshold applied across a feature that mixes shapes will be wrong in both directions at once: too tight where nothing is at stake, too loose where everything is.

Escalation is a route, not a shrug

Whatever the system declines has to arrive somewhere, and the quality of that arrival is most of what determines whether abstention is a feature or an outage.

A route needs a destination that exists in the organisation: a queue, an inbox or a shift with an owner, rather than a channel somebody muted. It needs a payload, meaning the input, the partial work, the signal that triggered the decline and a link back to the request. It needs a response expectation, because a queue with no time bound is a bin. And it needs a way back, so the human's answer reaches the waiting user.

The payload separates useful escalation from noise. A human who receives the document, the three fields the system did extract, and the note that the supplier name matched nothing in the vendor table can finish in under a minute. A human who receives a ticket saying the AI could not process this request starts from zero.

Escalation volume needs a ceiling with an alarm on it, for the same reason a cost budget does. A model change or an unusual week of inputs can turn a two per cent decline rate into thirty, and the first person to notice should not be the person whose queue filled up.

The decline is evaluated at the commit point

Chapter 4 argued for making the commit explicit so controls have somewhere to attach. The confidence gate is the clearest example of a control that belongs there.

There are two useful placements and they answer different questions. Gating before generation asks whether the request is one the system should attempt at all, which is cheap and catches out-of-range inputs before you spend anything. Gating at the commit asks whether this output should be allowed to have effects, which is where verification results and retrieval coverage are available.

Most features need both. The pre-gate keeps the bill down. The commit gate keeps the mistakes in.

One detail matters for auditing. Record the decline, the signal value and the route taken, in the same log that records commits. A system that logs what it did and not what it refused to do has discarded the more interesting half of its behaviour.

Over-abstention is the other failure, and it is quieter

The obvious risk in this chapter is a threshold that is too loose. The failure I see more often in features that took abstention seriously is the reverse.

A feature that declines twenty per cent of the time is not cautious. It is unreliable, and users respond to unreliability by building a habit that excludes it. They stop pressing the button on anything that looks hard, so the system only sees easy inputs, which makes the metrics look excellent while the value quietly leaves.

There is a structural reason abstention needs engineering rather than a prompt asking for honesty. Kalai, Nachum, Vempala and Zhang argue, in an arXiv paper published in September 2025, that language models hallucinate partly because training and evaluation reward guessing over acknowledging uncertainty: a model optimised for benchmarks graded right or wrong is rewarded for answering when unsure, since abstaining scores zero and a guess sometimes scores one. Their proposed remedy is to change how benchmarks are scored.

Take the product lesson rather than the research one. The instrument you hold was shaped by a regime that punished declining. So declining is not a behaviour you can request. It is one you impose from outside, with a gate.

A worked example, composited and labelled

The details here are composited from ordinary product shapes rather than one system, and the shape is exact.

A support product classifies inbound tickets into one of forty queues, then drafts a first reply. The original design used a self-reported confidence field with a threshold at 0.8 and routed everything below it to a general triage queue.

The rebuild replaces the score with three external checks. Classification abstains when the top two categories sit within a narrow margin across two samples, which is a disagreement signal rather than a confidence one. Drafting abstains when retrieval returns no policy passage covering the question, because a reply about a policy with no policy behind it was the specific failure they had shipped. Both abstain outright on languages absent from the evaluation set.

The routes were designed at the same time. Classification declines send the ticket to a triage rota with the two candidate categories shown, so the human clicks rather than reads. Drafting declines go to the queue for the category that was identified, with a note that no policy was found, which turns a failure into a knowledge-base gap somebody can close.

That gap list became the most useful artifact the feature produced. It did not require a better model.

The objection: a product that says it does not know looks weak

The commercial worry is real and deserves a direct answer rather than an appeal to honesty as a virtue.

The comparison is not between a confident system and an uncertain one. It is between a system wrong six times in a hundred with no warning, and one wrong twice in a hundred that flags six for a human. Users forgive the second and remember the first, because a wrong answer they acted on costs them something and a handoff costs them a wait.

There is a positioning argument too. Being able to state what the system does when it is unsure is a claim a competitor cannot make if they never built the state. Enterprise buyers ask the question directly.

Where the objection genuinely bites is on frequency and presentation. A decline that appears often, or that reads as an error, does damage. So keep the rate low by fixing the causes the decline log exposes, and present it as a routing decision rather than a fault: this one is going to a specialist, and here is the work so far.

Chapter summary

A confidence field the model writes into its own output is generated text rather than introspection, so a threshold set against it is a control in appearance only, and the ICML 2017 calibration result from Guo, Pleiss, Sun and Weinberger establishes the shape of the problem even though it concerns image classifiers: confidence and correctness are separate quantities that must be related empirically, by bucketing outputs and plotting observed accuracy per bucket. The usable signals come from outside the generator, in descending order of strength: verification against a system of record, which is boolean and cheap, agreement across independent samples or models, retrieval coverage meaning whether any supporting passage was found, and crude input-shape tells that place the request outside the range your error rate was measured on. Abstention is then a product state with a three-part contract covering what the user sees, what the system does not do, and which named queue receives the work, and it is distinct from a hedge, which asserts the content while shifting judgement onto the reader. Threshold placement is a business decision that trades the cost of a wrong committed answer against the cost of an unnecessary decline, and those costs are comparable for reversible actions and incommensurable for irreversible ones, which is why a single global threshold is wrong in both directions at once. Escalation needs a destination with an owner, a payload carrying the input, the partial work and the triggering signal, a response expectation and a path back to the waiting user, plus a volume ceiling with an alarm. Gates belong both before generation and at the commit point, and declines belong in the same log as commits. And over-abstention is the quieter failure, because a feature that declines often gets routed around, which leaves the metrics looking excellent as the value leaves.

Declining is one answer to a hard input. The other is to send it somewhere else in the system rather than out of it, which means picking a different model for a different step, on evidence rather than on habit. Chapter 6 is Routing: Capability, Cost, Fallback.