Abstention: Building a System That Can Say It Does Not Know

Most work on hallucination tries to make outputs more accurate. The complementary move, frequently more valuable and much less pursued, is to make the system decline when accuracy is unlikely.

The argument is straightforward once stated. A wrong answer delivered confidently costs more than no answer, because it is acted upon. An abstention costs the user a lookup they would have had to do anyway if the system did not exist. So for any claim type where being wrong carries real consequence, an abstention is strictly better than a guess.

Organisations resist this because abstention feels like failure and shows up badly in demos. A system answering ninety percent of questions correctly looks better than one answering seventy percent and declining thirty, even when the second is materially safer, because the refusals are visible and the errors are not.

That inversion is worth naming explicitly with stakeholders before deployment, because it determines whether the abstention mechanism survives its first review.

The technical problem underneath is that abstention requires knowing when the system is likely to be wrong, which is a question about uncertainty. The rest of this lesson is about why that is hard, and what actually works.

A model is calibrated if its stated confidence matches its accuracy: of the claims it asserts with seventy percent confidence, about seventy percent should be right.

Calibration is what would make abstention easy. Set a threshold, decline below it, done.

The difficulty is that the obvious way of obtaining confidence does not give you a calibrated one. Asking the model how confident are you produces a verbal statement generated by the same next-token process as the content. It reflects how confident such a statement typically sounds in this kind of context, not an internal quantity. Research examining whether stated confidence tracks actual reliability finds the relationship weak, and it can be systematically overconfident in exactly the situations that matter, where the model is generating under thin evidence.

Token probabilities are a better signal but not a solution. They describe the likelihood of a token given the context, which is related to but distinct from the truth of the claim the sentence makes. A fluent fabrication can have high token probability throughout.

So confidence has to be estimated externally rather than requested, which is what the next steps cover. The useful mental model: treat the model as an unreliable narrator about its own reliability, and build the measurement outside it.

Since asking does not work, uncertainty has to be inferred from observable properties of the generation and its context. Five signals, roughly in order of usefulness.

Retrieval quality. In a grounded system this is the strongest predictor available and the cheapest to compute. Low similarity scores on the top results, few results above threshold, or high disagreement between retrieved passages all predict that the model is about to improvise. This can be evaluated before generation, which means you can abstain without paying for the answer.

Self-consistency across samples. Variance between independent generations, from the previous lesson, is a genuine uncertainty signal, at the cost of several generations.

Entailment coverage. The proportion of claims in the output supported by sources. Computed after generation, cheap, and directly tied to the failure you care about.

Semantic dispersion. Where multiple samples exist, how far apart they are in meaning rather than in wording, since paraphrases should not count as disagreement.

And input-side signals: question type, entity obscurity, whether the question contains a false premise, and whether it asks for a claim type the first lesson flagged as risky.

The practical arrangement combines a cheap pre-generation signal, retrieval quality, with a cheap post-generation one, entailment coverage. That pair covers most cases without the cost of resampling.

Selective prediction formalises the trade. A system that may abstain has two numbers rather than one.

Coverage is the proportion of questions it answers. Accuracy is how often those answers are right. Raising the abstention threshold lowers coverage and raises accuracy on what remains, and the relationship between them is the curve that describes your system.

That curve, not any single point on it, is what should be presented when deciding a threshold. It shows what accuracy is purchasable at what cost in coverage, and it makes the decision a business one rather than a technical default.

The threshold then follows from consequence. Where a wrong answer is expensive and a lookup is cheap, sit high on the accuracy side. Where the task is exploratory and errors are caught downstream, sit low.

And it should differ within one system. A single global threshold applied to both a brainstorming question and a dosage question is wrong for at least one of them.

Threshold selection is usually done by eye, which leaves you unable to say anything precise about the resulting error rate. Conformal prediction offers an alternative with a stated guarantee.

The approach, applied to this problem by Yadkori and colleagues in work presented at NeurIPS in 2024 under the name conformal abstention, works as follows. Take a calibration set of questions with known correct answers. Compute your uncertainty score on each. Use the distribution of scores on that set to choose a threshold such that, on data from the same distribution, the error rate among answered questions is bounded at a level you specify.

What that buys is a statement of the form: on data like the calibration set, the hallucination rate among answers given will not exceed this bound, with this probability. That is qualitatively different from a threshold someone picked because it looked reasonable, and it is the kind of statement the regulated-industries cursus needs for an evidence pack.

Two caveats bound the claim. The guarantee holds under exchangeability, meaning production data resembles the calibration set, and that assumption degrades as usage drifts, which is why calibration has to be refreshed. And a calibration set needs known correct answers, which for open-ended generation is expensive to build and is usually the binding constraint rather than the method.

Still, for a system with a well-defined task, this converts an arbitrary threshold into a defensible one.

A technically correct abstention that frustrates users gets removed. The interaction design determines whether the mechanism survives.

Refuse specifically, not generically. I cannot help with that teaches nothing. The documents I searched do not cover the 2024 policy revision tells the user what happened and what to do next.

Offer the next step. Show the closest material found, name who owns the topic, or suggest a reformulation. An abstention that hands over something useful is a service rather than a dead end.

Distinguish the reasons, because they warrant different responses. Not in the corpus, ambiguous question, outside permitted scope, and low confidence in an available answer are four different situations, and collapsing them into one refusal message wastes information the system already has.

Use partial abstention. Full refusal is rarely necessary. Answer the parts that are supported and mark the parts that are not, which preserves most of the value while removing the unsupported claims.

And make the confidence visible rather than binary where the interface allows. A clearly marked provisional answer, flagged as unverified, lets the user apply their own judgement, and users handle acknowledged uncertainty far better than they handle unmarked error.

The principle underneath: abstention is a communication problem as much as a modelling one, and most of the resistance to it is caused by poor communication rather than by the refusals themselves.

Everything above needs measurement, and hallucination is unusually hard to measure in production because the ground truth is not delivered by the system.

Four practical approaches, none sufficient alone.

A fixed evaluation set with known answers, run on a schedule. Gives a comparable number over time and catches regressions, particularly after a model version change. Its weakness is that it ages and stops representing real usage.

Sampled human review of production outputs, scored against sources by someone competent. Expensive, and the only method that measures what users actually receive. Sample by risk rather than uniformly.

Automated groundedness scoring on live traffic, using entailment coverage as a continuous proxy. Cheap enough to run on everything, imperfect, and valuable as a trend rather than as an absolute.

And downstream signals: user corrections, thumbs-down, escalations, complaints, and cases where a human overrode the output. These are the cheapest and the most lagging.

Two cautions. Do not report a single hallucination rate across a system doing several tasks, since it is an average of incomparable things. And track abstention rate alongside accuracy always, because accuracy improving while abstention climbs may just mean the system stopped answering the hard questions, which is a different result from getting better at them.

Some contexts make abstention non-negotiable, and identifying them early avoids a late and expensive redesign.

Where the user cannot verify. A system answering questions in a domain the user does not know cannot rely on them catching an error, so the system must catch it. This is the majority of internal knowledge assistants and it is the case most often overlooked, because the deployment rationale is precisely that users lack the expertise.

Where the answer triggers an action. Anything feeding an automated decision or a tool call, since there is no human between the fabrication and the consequence.

Where the claim type is high-consequence: dosages, legal obligations, financial figures, safety procedures, regulatory deadlines. Abstain per claim type rather than per question.

Where the system will be cited. Output that becomes an input to a document or another system propagates errors beyond the point of generation.

And where a regulator will ask. In the terms of the regulated-industries cursus, a system that always answers has no evidence of judgement, and the abstention rate is itself evidence that the system distinguishes what it can and cannot support.

If a deployment sits in any of these and has no abstention path, the honest description is that the residual hallucination rate is being absorbed by whoever receives the output, and that should be a recorded decision rather than an unexamined default.

The design that follows from all three lessons, in the order the decisions should be made.

Decide what the system must not get wrong, by claim type rather than globally. That list drives everything else.

Ground it, so there is something to check against. Measure retrieval separately from generation, because a large share of what looks like hallucination is retrieval failure.

Give the model permission to fail, explicitly, in the prompt. This is nearly free and changes behaviour more than most interventions that cost real money.

Verify at claim level, with cheap entailment checks on everything and expensive methods targeted at the high-consequence claim types.

Abstain on the signals, with a threshold chosen from the coverage-accuracy curve by consequence, calibrated properly where the task is well-defined enough to permit it.

Communicate the abstention specifically, with a next step, and prefer partial abstention to refusal.

Measure continuously, on a fixed set and by sampled review, tracking abstention rate alongside accuracy.

And accept a residual rate, because the first lesson established that one exists regardless. Record what it is, who bears it, and what happens when it materialises.

That last step is the difference between a system whose limits are understood and one whose limits will be discovered by a user.