RLVR is a Kernel, Not a Function: Statistical Inference for pass@$k$ Crossovers
Chen Yang, Chen Yang, Xianyang Zhang, Jun Chen
stat.ML · cs.LG
Quick verdict
A rigorous, pre-registered paired-inference framework for pass@k crossovers; only one small, token-limit-dependent reversal is statistically confirmed.
Summary
This paper re-examines the RLVR 'sharpening' debate statistically. The authors build a paired dominance test and simultaneous (sup-t) confidence bands for the pass@k difference curve across sampling budgets k, with a pre-specified crossing criterion requiring both an early gain and a later loss. Applied to five public checkpoint pairs, no crossing is statistically established in initial evaluations; the strongest case (DeepScaleR at a 16k token limit, p=.029) fails Holm correction and does not replicate on fresh prompts, while a pre-registered 32k-token experiment on 1060 fresh prompts confirms a crossing with first loss between k=11 and 61 — though the late-loss margin is very small. A second contribution is that the prompt-level response to training is a conditional distribution — a Markov kernel — rather than a deterministic p→q map: a split-generation residual test rejects determinism on all pairs and non-Qwen bases, tail-exponent theorems characterize when inversions are forced, and collapsing the fitted kernel onto its conditional mean removes crossings within the observation window. Controlled GRPO-versus-SFT runs show the 'opposite sides of slope one' story is an affine-link artifact, and power analysis recommends breadth (more prompts) over depth. Main limitations: small effect size, mixture-specificity, token-limit sensitivity, and validation on held-out generations rather than new prompts.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- No public RLVR-vs-base pair has a statistically established pass@k crossing in initial evaluations: the strongest dominance rejection (DeepScaleR, 16k, p=.029) is not family-wise significant under Holm and fails a fresh-prompt check at the same limit (p=.91).
- A pre-registered 32k-token evaluation on 1060 fresh math prompts confirms one crossing for DeepScaleR vs base, with first loss between k=11 and 61 and a late-loss margin roughly ten times smaller than the early gain.
- The prompt-level response to RLVR is a Markov kernel H(q|p), not a deterministic function of base success: a split-generation residual covariance test rejects determinism on all thirteen pairs and two non-Qwen bases.
- Collapsing the fitted kernel onto its conditional mean removes every observed-window crossing, so the measured inversions are driven by conditional dispersion rather than by the tail-elasticity mechanism of Theorem 2.
- At realized designs, kernel-simulated dominance-test power spans .01–.63, breadth (more prompts) beats depth at equal cost, and failure to detect a crossing is not evidence of its absence.
Strengths
Formally grounded, layered paired inference
Theorem 1 gives a multiplier-bootstrap dominance test with explicit size control at the all-binding boundary (D≡0) and a pathwise inequality argument; Proposition 1 turns a sup-t simultaneous band into a crossing criterion with a first-loss interval [κ̲, κ̄] (Table 1: κ∈[11,61] at 32k fresh). The paper's four-layer claim taxonomy (visual change / dominance violation / supported crossing / confirmed protocol) directly targets the ambiguity in prior crossover readings.
Discovery-to-confirmation discipline with committed negative outcomes
Appendix G pre-specifies decision rules and treatments for every outcome before sampling; Table 1 separates discovery (16k), a failed same-limit fresh check (p=.91), an adaptive 32k mixture follow-up, and the pre-specified 1060-fresh-prompt confirmation (dominance p=.0027, pass). Holm correction is applied across the five-pair discovery family, and the strongest initial finding (p=.029) is explicitly reported as not family-wise significant.
Sensitivity and robustness work well beyond field norms
The kernel/dispersion finding is tested, not asserted
Honest theory–mechanism separation
Weaknesses
The single confirmed crossing is small and population/protocol-bound
Kernel validation holds out generations, not prompts
Scale comparison confounded by design
Dense, hedge-heavy presentation
Several quantitative claims rest on truncated appendix material
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper targets the RLVR-capability debate inaugurated by Yue et al. (2025): if an RL-trained model beats its base at pass@1 but loses at pass@256, does that prove RLVR merely 'sharpens' existing capability? The authors' answer is methodological: the crossover picture itself has never been statistically established. They build (i) a paired dominance test (Theorem 1) with multiplier-bootstrap critical values, (ii) sup-t simultaneous confidence bands over sampling budgets k, and (iii) a crossing criterion (Proposition 1) requiring L(j)>0 at some j and U(ℓ)<0 at some ℓ>j, plus a first-loss interval κ∈[κ̲, κ̄]. Four layers of evidence are kept separate throughout: visual sign change, dominance violation, supported crossing, and confirmation under a new protocol.
Two headline results. First, across five public pairs (Table 1), no crossing is established initially: ProRL's k=56 crossing gives p=.20; DeepScaleR-16k's p=.029 fails Holm over the five-pair family (threshold .01) and fails a fresh-prompt check at the same limit (p=.91). A pre-registered 32k run on 1060 fresh prompts (OlympiadBench/Gaokao/AMC23/AIME24, n=n′=128) passes: L(k)>0 through k=10, U(k)<0 from k=61, κ∈[11,61], dominance p=.0027 — but with δ−=3.7e-3 against δ+=.0655. Second, the p→q operator is a Markov kernel, not a function: a split-generation residual covariance test (Table 5) rejects determinism on all thirteen pairs (calibrated p≤.002; RMS dispersion .10–.15 vs ≈.03 binomial noise), including Llama-3.1-8B and DeepSeek-Math-7B bases.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does the paper claim about RLVR pass@k crossover evidence?
It claims that published 'the curves cross' evidence has not been statistically established: across five public checkpoint pairs, no crossing meets a criterion requiring both a significant early gain and a significant later loss in initial evaluations. One crossing (DeepScaleR vs its base) is confirmed in a pre-registered 32k-token evaluation on fresh prompts, with first loss between sampling budgets k=11 and 61, but the effect is small and specific to that token limit and prompt population.
Why does the 32k token limit matter for pass@k comparisons?
Because pass@k is computed from generations that stop when a token cap is hit, the cap acts as a censoring mechanism that can manufacture or erase an apparent crossover. In the paper's Figure 1, DeepScaleR vs base shows no crossing at an 8k cap, an unconfirmed sign change at 16k, and a statistically supported crossing only at 32k, so comparisons should report truncation rates and test limit sensitivity.
What is the Markov kernel finding in this paper?
The paper shows that the effect of RLVR training on a prompt is not a fixed function of its base success probability: prompts with the same base success rate get systematically different post-training success rates, reproducibly across independent generation halves. It models this as a conditional distribution H(q|p) — a Markov kernel — validated by a split-generation residual test that rejects the deterministic map on all thirteen pairs and on non-Qwen bases.
Should evaluators sample more prompts or more samples per prompt?
According to the paper's power analysis, more prompts at a given depth is usually the better spend, because between-prompt variability does not shrink as samples per prompt grow: at equal cost, DeepScaleR-16k reaches power .877 with m=4000, n=128 versus .489 with m=1000, n=512 (Holm threshold). The caveat is that the per-prompt count n must cover the target budgets k≤n first.
Is the confirmed crossing evidence that RLVR does not add capability beyond the base model?
No. The paper explicitly separates evidence that a crossover exists from what it means for capability: its theory shows losses on a minority of hard prompts can overturn an early lead even when most prompts improve, and the confirmed late-loss margin is very small (about ten times smaller than the early gain).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The extracted text is incomplete: §5's Finding 4 listing, kernel refit details (E.2), Table 8's surroundings, and parts of Appendix H fall in omitted spans; judgments about those pieces rely on cross-references in reproduced sections.
- Figures (1, 4, 5) are available only via captions and quoted values; band shapes were not directly inspectable.
- All numeric claims are taken from the provided text; independent verification of code, counts, or proofs (Appendices A–H) was not performed.
- The appendix table of contents was cut mid-way (at B.2), so some referenced appendix material (e.g. Tables 3, 5 context, 6) is only partially visible.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.