The Missing "I Don't Know": Why Three Reasoning-Reliability Findings Converge on Calibrated Abstention
cs.LG · cs.AI · cs.CL
Quick verdict
Position paper: three LLM reliability findings converge on calibrated abstention; four benchmark reforms proposed. Well-hedged but no new experiments.
Summary
This position paper argues that three recent LLM-reliability results — Yin et al.'s (2026) finding that reasoning RL collapses tool-reliability representations, Suleymanov et al.'s (2026) rewrite-versus-truncate divergence under safety-constrained generation, and Bastounis et al.'s (2024) theorem that consistent reasoning without an 'I don't know' function must hallucinate infinitely often — converge on calibrated abstention as the missing capability, despite differing sources of unavailability (capability, policy, recursion-theoretic gaps). The author contends that dominant benchmarks (MMLU, HellaSwag, HumanEval, GSM8K) assign zero reward to decline, so the leaderboard-level selection signal for abstention is absent. Four reforms are proposed: triple-scoring (formalized as Chow's 1970 cost-sensitive abstention threshold with a p* sweep and a proper score on committed answers), abstention-rate reporting, capability-stratified evaluation, and mandatory calibration metrics. A re-scoring of published SimpleQA outcome rates (Table 4) illustrates rank inversions between accuracy and utility orderings, with an always-abstain null policy dominating all models. Strengths include explicit defeasibility and falsification conditions and a four-construct abstention taxonomy. Limitations: no new experiments; the necessity claim rests on interpretive synthesis plus a proxy-based falsification condition; and the Table 4 demonstration uses grader-derived 'not-attempted' labels rather than a structured abstain action.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
4 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Three independently-sourced reliability findings (a capability gap, a policy gap, and a recursion-theoretic gap) converge on calibrated abstention as the missing capability, though they are explicitly not instances of Bastounis's theorem.
- Dominant discrete-answer benchmarks assign zero reward to decline, so the leaderboard-level selection gradient that would favor abstention-capable models does not exist where model releases are ranked.
- Four evaluation reforms — triple-scoring, abstention-rate reporting, capability-stratified evaluation, and mandatory calibration metrics — form a necessary-but-not-sufficient bundle for closing the reasoning-reliability gap the theorem identifies.
- Honesty post-training narrows the gap only where the model itself recognizes uncertainty; on the complementary slice (confabulation where the model believes it knows) it does not — offered explicitly as a testable conjecture, not a finding.
- The proposed reward inequality is Chow's (1970) cost-sensitive abstention threshold applied where abstain was priced at zero; the claimed contribution is the composed bundle with a p* sweep and proper-score verification, not the rule itself.
Strengths
Explicit defeasibility and falsification conditions
Sections 1, 3, and 6 state precisely what would refute the convergence reading (a mechanism-level, non-abstention intervention closing all three failure modes at once) and the necessity reading (sub-10% residual error on an abstention benchmark stratified by environment-level unanswerability). This is unusual epistemic hygiene for a position paper.
Honest scoping of novelty and formal claims
The paper states the scoring rule is Chow (1970) and 'not claimed as new' (Section 5), explicitly disclaims that practical abstention instantiates Bastounis's recursion-theoretic object and that the theorem is not applied to LLM inference directly (Section 2.3), and separates four abstention constructs (Table 1), avoiding the common overreach of applying an impossibility theorem to current models.
Concrete quantitative demonstration
Considered boundary conditions and second-order effects
Weaknesses
No new empirical evidence for the central convergence claim
Load-bearing reliance on fragile secondary numbers
Flagship demonstration does not instantiate the proposal
Necessity claim anchored to a proxy
Contestable benchmark premise
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the main argument of this paper?
It argues that three recent LLM reliability findings — reasoning RL increasing tool hallucination (Yin et al.), safety-constrained generation causing large models to rewrite while small models truncate (Suleymanov et al.), and a theorem that consistent reasoners without an 'I don't know' function must hallucinate infinitely often (Bastounis et al.) — all point to the same missing capability: calibrated abstention. Because major benchmarks give zero reward to declining to answer, no leaderboard pressure selects for this capability, so the paper proposes four evaluation reforms: triple-scoring, abstention-rate reporting, capability-stratified evaluation, and mandatory calibration metrics.
Does the paper include new experiments or code?
No. It is a position/analysis paper with no new data collection. Its one quantitative demonstration re-scores previously published SimpleQA outcome rates under two abstention-price settings to show that accuracy rankings and abstention-aware utility rankings diverge, and that a model that always abstains would top the re-scored board. No code or artifacts are released.
Is the 'I don't know' impossibility result new?
No. The theorem is due to Bastounis et al. (2024), and the abstention scoring rule the paper uses is Chow's 1970 cost-sensitive reject threshold. The paper explicitly disclaims novelty for both and states its contribution is the argument that these pieces, composed with three additional reporting reforms, form a structural requirement rather than optional extras.
What would make the authors change their position?
The paper names two exit conditions. First, if a mechanism-level intervention that implements no abstention or selective-prediction construction closed all three documented failure modes at once, the convergence reading is refuted. Second, if post-training alone achieved sub-10% residual error on a calibrated abstention benchmark stratified by environment-level unanswerability, the authors would withdraw the claim that benchmark reform is structurally necessary and treat it as a complement.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The three source papers are assessed only as characterized within this paper; their contents were not independently verified.
- Several load-bearing citations are 2025-2026 arXiv preprints whose peer-review status and possibly revised headline numbers could not be checked.
- Tables were extracted linearized from PDF; minor transcription risk applies to Table 4 numbers and ranks.
- The paper contains no figures (tables only), so nothing figure-dependent could be analyzed.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.