Wrong Prediction, Right Answer: Recovering Evidence from Collapsed LLM Sequence Scores
Qiyao Yan, Chenpeng Wang, Liangming Pan
cs.AI
Quick verdict
Rigorous diagnostic: a 2-parameter label-free offset rescues collapsed LLM sequence scores on balanced 3-way tasks; scope-bounded, minor overclaiming.
Summary
The paper argues that a wrong final prediction does not imply a model lacks reasoning ability: on several three-way reasoning benchmarks, summed candidate-answer log-probabilities collapse onto a single label (Qwen3.5-9B predicts 'unknown' for 999 of 1,000 lexical-OOD examples) while linear probes on hidden states decode the correct answer at up to 0.972 in-domain. Localizing the failure at the unembedding and sequence-aggregation stages, the authors introduce a diagnostic that adds one constant offset per label — two free parameters — fitted by grid search on 25–1,000 unlabeled in-domain scores to match a supplied uniform target prior, using no ground-truth labels. On controlled logic, ProofWriter, ANLI R2, and FOLIO, this lifts Qwen3.5-4B/9B from chance to 0.570–0.678, with 25-example fits positive in 30/30 subsamples; OLMo-2-1B (+20.4 points) and Llama-3.1-8B (+14.4) transfer on ProofWriter, while Pythia shows no reliable recovery, marking the scope boundary. Controls include TF-IDF-missed slices (0.622–0.643 on hard ProofWriter), count-preserving permutation nulls (+0.287/+0.305 gaps), prior perturbation, verbalizer grids, and seed regeneration. Main limitations: the uniform prior is supplied and all splits are balanced; recovery remains partial (0.57–0.60 versus probes at 0.77–0.83); there is no head-to-head comparison with existing calibration methods; and the abstract's '9–34 point' range omits smaller 2B gains (e.g., +4.8 points).
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Hidden-state probes decode correct answers while native sequence scoring stays at chance on three-way reasoning tasks, for both post-trained and Base Qwen3.5 checkpoints, locating the failure at the unembedding and sequence-aggregation stages.
- A two-parameter, target-label-free additive offset fitted on as few as 25 unlabeled examples recovers up to 34 accuracy points on Qwen3.5 models and transfers to OLMo-2-1B and Llama-3.1-8B on ProofWriter.
- Recovered decisions are per-example rather than histogram artifacts: they persist on TF-IDF-missed slices and significantly exceed count-preserving permutation baselines.
- Recovery has delimited boundaries — Pythia and near-ceiling configurations show no reliable gain — so the offset is a diagnostic for collapsed-but-informative rankings, not a universal fix.
- Benchmark errors should be diagnosed across internal representations and answer scoring before being attributed to missing reasoning capability.
Strengths
Unusually complete control battery isolates per-example recovery
Count-preserving permutation nulls (gaps +0.287/+0.305 on ProofWriter, Table 4), TF-IDF-missed slices (0.622–0.643 at 25-example budget, Table 26), worst-case prior perturbation still positive (+0.193/+0.242 synthetic, Table 19), 20 prompt/verbalizer variants, equal-token ABC verbalizers (Table 9), mean-normalization (Table 21), and 3 independent generator seeds (min delta +0.230, Table 23) together rule out histogram-matching, lexical-shortcut, and normalization artifacts.
Clean stage-localization of the failure
Position-matched comparison of prompt-end probes, answer-slot probes, same-position frozen logits, and full string scores on identical 1,000-example splits (Table 1, 11) shows information lost at the unembedding and again at sequence aggregation: Qwen3.5-9B-Base label logits reach 0.627 (ID) while the string score re-collapses to 0.374 — a genuinely informative structural finding.
Honest boundary conditions
Artifact-free probe evidence
Sample efficiency with full uncertainty reporting
Weaknesses
Overclaiming language inconsistent with the paper's own hedging
Oracle prior on balanced splits only
No head-to-head against existing calibration baselines
Headline range does not match reported minima
Recovery is partial and the residual gap is unexplained
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper attacks a common inference in LLM evaluation: that a wrong final answer means absent reasoning capability. Its positive claim is a two-part finding. First, a *readout gap* (Section 4.2): on three-way tasks (true/false/unknown), linear probes on hidden states decode answers at 0.918–0.972 in-domain for Qwen3.5-2B/9B while native full-string scores sit at exactly 0.333, and the gap survives on Base checkpoints (Table 11), ruling out an instruction-tuning artifact. Second, a *recovery claim* (Sections 4.3–4.5): a two-parameter, target-label-free additive offset fitted on as few as 25 unlabeled examples lifts collapsed scores substantially, and the recovered decisions are per-example rather than histogram artifacts. The evidence base is deliberately narrow — balanced three-label forced-choice tasks (controlled logic, ProofWriter, ANLI R1→R2, FOLIO) on 0.4B–12B open-weight models — yet the abstract asserts the results 'definitively show' that 'many apparent zero-shot reasoning deficits are merely expression failures.' That universality outruns both the design and the paper's own Section 5, which reframes the offset as 'a descriptive diagnostic... not a universal causal fix.'
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does 'collapsed sequence scoring' mean in this paper?
On some three-way reasoning tasks (true/false/unknown), the model's summed log-probabilities over candidate answer strings pick the same label for nearly every question — for example, Qwen3.5-9B predicts 'unknown' for 999 of 1,000 lexical-OOD examples — so accuracy sits at chance (~33%) even though linear probes on hidden states decode the correct answer far above chance (up to 0.972 in-domain).
What is the proposed fix, and does it require training the model?
No training. The method adds one constant offset per candidate label (two free parameters for three labels), chosen by a small deterministic grid search so that the prediction distribution on 25–1,000 unlabeled in-domain examples matches a supplied uniform prior. No ground-truth labels are used, and the model's weights are frozen.
Does this prove that failing LLMs can actually reason?
Not universally. The authors show that for some models (Qwen3.5 family, OLMo-2-1B, Llama-3.1-8B) correct answers remain encoded internally and survive in native sequence scores once a label-level bias is removed, but Pythia checkpoints show no reliable recovery, and corrected accuracy (~0.57–0.60 on shifted tasks) stays well below probe accuracy — the paper itself frames the result as diagnostic evidence, not proof of reasoning.
Is code or data available for this paper?
The study uses public benchmarks (ProofWriter, ANLI, FOLIO), a synthetic controlled-logic generator with fixed seeds, and open-weight checkpoints with pinned HuggingFace revisions; the text mentions submission artifacts (manifest and CSVs), but no public code repository URL appears in the paper, so reproduction currently requires reimplementing the scoring and offset-fitting pipeline.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure panels (Figures 1–4) are extracted as interleaved, garbled embedded text; figure-level judgments in this review rely on captions and tables, not rendered panels.
- Appendix ordering is jumbled by PDF extraction (e.g., Tables 43–45 appear before Section H); every table cited in this review was verified present in the provided text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.