Proof-Carrying Cognition: Closing the Verification Gap with Reality-Settled Reward
Eshwar Reddy M, Sourav Karmakar
cs.AI · cs.LG
Quick verdict
Claims reality-settled reward curbs verifier gaming; pre-registered, unusually honest evidence at toy+API scale, but the paradigm stays untested at RL scale.
Summary
The paper argues that the binding constraint on reasoning RL is the 'verification gap': no scalable, incorruptible reward exists outside formally verifiable domains. Theory: in best-of-N selection under a joint-Gaussian model, verifier-gold correlation ρ is the exact exchange rate between test-time compute and capability, with an unsound verifier paying N^(1/ρ²) candidates; the authors themselves show the closed form fails outside the model class and repair it with an exact finite-N form (3.4% error) and a margin-free copula law predicting real-judge soundness to 4.1% median error. Demonstrations: in six- and ten-token DSL testbeds with a pre-registered replication, the weak verifier loses Soundness-under-Pressure (0.94→0.32 at N=4096) while a sound verifier improves monotonically; reality-anchored settlement beats frozen verifiers under adversarial pressure (gap ~0.27→~0); on-policy settlement is ~10× more label-efficient. Real-model experiments (claude-haiku-4-5 generator, weak/strong judges, executed MBPP/HumanEval tests as gold) show the weak judge degrading under selection (p<10⁻³) and selection alone manufacturing +0.53 hacking gaps. In a GRPO run, a frozen reward model's executed reward collapses 90% while a twin refit on a 10% settlement stream retains 6× more. The paper proposes proof-carrying cognition and a Soundness-under-Pressure benchmark. Limitations: toy synthetic domains, a saturated real-code suite (n=60/18, N≤32), one model family, a failed drift-alarm criterion, and a paradigm untested at RL training scale.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- In best-of-N selection under a joint-Gaussian model, verifier-gold correlation ρ is the exact exchange rate between test-time compute and capability; outside that model class only the qualitative ordering survives, with a margin-free copula form predicting realized soundness of real LLM judges to 4.1% median error.
- Learned verifiers lose Soundness-under-Pressure as optimization pressure grows (weak verifier 0.94→0.32 at N=4096), while reality-anchored settlement bounds or inverts reward hacking (adversarial gap ~0.27→~0).
- Selection pressure over honest samples, not an adversarial generator, is the operative Goodhart force on LLM judges: mining max-(judge−gold) candidates from 32 honest samples yields +0.527 gap vs −0.209 for instructed deception.
- Under live GRPO, a frozen reward model's executed reward collapses ~90% (peak 0.617→0.063) while an identical RM refit on a 10% execution-settlement stream retains 6× more executed reward and beats an equal-budget LLM-judge-labeled control, isolating reality as the label source.
- Soundness-under-Pressure (area under Snd@N over log N) should be the headline metric of a reality-settled reasoning benchmark (RSR-Bench), which the authors call the field's most important near-term investment.
Strengths
Pre-registration with honest failure reporting
§7 Table 2 reports 10 registered hypotheses as 6 pass / 3 fail / 1 invalid; the authors disclose that two criteria (H2b, RL3c) were themselves statistically mis-specified, that the toy suite's inverted hacking gap did not reproduce at scale, and that the drift alarm (R-D) failed its first live test — an unusually high level of self-falsification for arXiv.
Theory falsified and repaired inside the same paper
H1b/H1c failures led to concrete repairs: the exact finite-N penalty matching empirical budgets within 3.4% (vs the N^(1/ρ²) asymptotic overstating by up to 24×), and the margin-free copula law (Prop. 2) predicting real-judge realized soundness with 4.1% median error where Pearson ρ errs by 0.793.
A clean label-source control in live RL training
Actionable negative and mechanism findings
Explicit cost accounting of trust
Weaknesses
The titled paradigm is never instantiated
Fragile real-model suite
Core safety mechanism lacks positive evidence
Headline efficiency claims are conditional
Multiplicity and provenance gaps
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
Self-described hybrid position-and-proof-of-concept paper (§1). Thesis: RL gains concentrate where cheap, sound verifiers exist; elsewhere the binding constraint is the verification gap. Contributions: (1) an exchange-rate theory (Prop. 1: E[G(i*)]=ρE[max_i P_i]; Cor. 1: N^(1/ρ²) penalty); (2) a demonstration ladder — minimal DSL (§6), pre-registered scaled replication (§7), real-code/real-model suites (§8), live GRPO training (§8.2); (3) the proof-carrying-cognition paradigm (Algorithm 1); (4) the Soundness-under-Pressure metric and RSR-Bench recipe (§9). The paper is explicit about its own failures: the replication falsified quantitative transfer of the exchange rate (H1c) and the closed-form penalty (H1b), and the drift alarm failed its first registered test (R-D).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the 'verification gap' in this paper?
The absence of a scalable, incorruptible reward signal for reasoning outside formally checkable domains (proof kernels, code execution, game rules). The authors argue that human preference labels, reward models, and LLM judges all become gameable under enough optimization pressure, which is why RL-trained reasoning gains have concentrated in formally verified domains like mathematics and code.
What is 'proof-carrying cognition' (PCC)?
A proposed training paradigm in which a reasoner emits reasoning steps as typed probabilistic claims; a self-built world model prices the claims densely, reality (execution traces, experiments, later events) settles them sparsely but incorruptibly via strictly proper scoring rules, and the world model is trained only on settled reality — making reality, rather than human or judge approval, the ultimate reward function.
Did the paper's experiments support its claims?
Partly. In the pre-registered scaled replication, six of ten hypotheses passed; the quantitative Gaussian exchange-rate law failed to transfer outside its model class (repaired by a copula form with 4.1% median error), the closed-form N^(1/ρ²) penalty overstated costs at practical N (exact finite-N form within 3.4%), and the drift-alarm criterion failed its first live GRPO test — all reported by the authors themselves.
What is Soundness-under-Pressure (Snd@N)?
A proposed benchmark metric: the ratio of the expected gold reward of the candidate a proxy verifier selects from N samples to that of the gold-selected candidate. A sound verifier scores 1 at every N, and the headline score is the area under Snd@N over log N, measuring verifiers under the selection pressure they actually face rather than at a single accuracy point.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text is a sampled excerpt with '[...sampling gap...]' markers; parts of §4, §8.2 registration text, and §10 were skipped, so judgments touching those sections rest on the visible fragments.
- The extraction contains duplicated blocks (repeated figure captions and section passages, e.g., §6.7/§7 and Figures 3–5); these are extraction artifacts and were read deduplicated, not treated as paper defects.
- The claimed public release of code, seeds, pre-registration, and raw data could not be verified from the provided text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.