Reflective Recovery: A Self-Supervised Method for Reasoning by Learning from Mistakes
Qirui Chen, Renjie Pi, Jiahui Gao, Lingpeng Kong
cs.CL
Quick verdict
Turns failed LLM reasoning traces into recovery training data; consistent but modest gains over RFT, with gaps in the scaling-collapse evidence.
Summary
The paper proposes Reflective Recovery, a fine-tuning pipeline that converts a model's failed reasoning trajectories into training data for error recovery. From 3k mathematical seed problems, the authors sample eight trajectories per problem with a rule-based answer verifier, split them into correct (Dpos) and failed (Dneg) sets, truncate each failed trajectory at an intermediate step, resample five continuations, and keep only completions that reach the verified answer, yielding a recovery set Drec. Models are then fine-tuned with loss applied only to the resampled continuation, keeping the erroneous prefix as masked context. On DeepSeek-R1-Distill-Qwen-7B/14B, the method outperforms data-matched rejection-sampling fine-tuning (RFT) across seven benchmarks, e.g., AIME 2025 rising from 34.7% to 37.5% (7B) and Minerva from 47.4% to 50.7% (14B). The authors argue positive-only imitation suffers a 'scaling collapse' (RFT drops from 35.3% to 30.0% on AIME 2025 as data grows from 8k to 10k) while their method keeps improving. Ablations cover truncation position, a second recursive data iteration, and hybrid RFT+RR training. Main limitations: RFT is the only baseline, no statistical uncertainty is reported for small deltas, the key control (truncating correct trajectories) is missing, and Figure 2's scaling evidence cannot be verified from the text.
Review complete
Assessment at a glance
4
risk flags identified
● high 1
● medium 2
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Positive-only imitation learning (RFT) suffers a 'scaling collapse': on AIME 2025 its accuracy drops from 35.3% to 30.0% as training data grows from 8k to 10k trajectories.
- Fine-tuning on resampled continuations of truncated failed trajectories (Reflective Recovery) outperforms data-matched RFT across seven benchmarks on DeepSeek-R1-Distill-Qwen-7B and 14B.
- Unlike RFT, the method scales stably with more data and avoids the saturation/degradation of positive-only imitation.
- The gains reflect genuine mid-process self-correction, evidenced by more frequent reflective language patterns and by benefits from recursive second-iteration data.
Strengths
Fully specified, critic-free pipeline
The Truncate-Resample-Select procedure (§3.2) uses only a rule-based \boxed{}-answer verifier (rel_tol 1e-5); loss masking of the erroneous prefix (§3.3) is a clean mechanism against reinforcing mistakes, and prompt/training formats are given in Appendix B.
Data-matched controlled comparison
|Dpos| = |Drec| at 6k for main results (§4.1), identical 5-epoch Open-R1 recipe for both methods, and gains in the same direction across all seven benchmarks and both model scales (Table 1).
Unusually complete ablation set
Strong reproducibility inputs
Weaknesses
Single baseline limits the comparative claim
Missing control isolating the role of errors
No statistical uncertainty anywhere
Scaling narrative in tension with own ablation
Mechanism evidence is correlational
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This method paper targets supervised fine-tuning for LLM mathematical reasoning. The authors make three claims: (1) positive-only imitation learning (RFT) suffers a "Scaling Collapse" on limited problem sets — on AIME 2025, RFT accuracy falls from 35.3% to 30.0% as training data grows from 8k to 10k trajectories (§1, §4.3, Figure 2); (2) Reflective Recovery (RR), a pipeline converting failed trajectories into error-prefix-plus-correct-continuation pairs, beats RFT at matched data volume across seven benchmarks on DeepSeek-R1-Distill-Qwen-7B/14B (Table 1); (3) the method "breaks the scaling collapse barrier" and elicits "emergent self-correction." Claims (1) and (2) are supported by the reported tables; claim (3) is only partially supported and rests partly on Figure 2, which cannot be inspected from the extracted text.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Reflective Recovery in LLM training?
It is a fine-tuning data pipeline that takes a model's own failed reasoning chains, cuts them at an intermediate step, and resamples continuations until one reaches the verified correct answer. The model is then fine-tuned to produce correct continuations given erroneous prefixes (with the prefix loss-masked), teaching mid-reasoning error recovery without any reward model or reinforcement learning.
How much does Reflective Recovery improve over standard RFT?
At matched data volume (6k trajectories), the paper reports AIME 2025 at 37.5% vs 34.7% for RFT and Minerva at 47.8% vs 46.7% on DeepSeek-R1-Distill-Qwen-7B, with consistent direction of improvement across seven benchmarks and larger gains on the 14B model (e.g., Minerva 50.7% vs 47.4%).
Does Reflective Recovery need a reward model or reinforcement learning?
No. The only supervision signal is a rule-based answer checker that extracts \boxed{} answers and compares them to ground truth with numerical tolerance; no external critic, reward model, or RL stage is used, though labeled problems with known answers are required.
Is code or data available for this paper?
No code or data release is mentioned in the paper. However, all components are public — DeepSeek-R1-Distill-Qwen base models, Light-R1 seed problems, the Open-R1 training framework, and the lighteval evaluation harness — so an independent re-implementation is feasible, though 32k-context SFT on 7B/14B models requires multi-GPU resources.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-6 are not analyzable from the extracted text; the central scaling evidence (Figure 2) and the reflective-pattern counts (Figures 3-4) could not be independently checked.
- The results table header lists two LiveCodeBench columns ('LiveCodeBench' and 'LiveCodeBench V4'); their exact versions/relationship cannot be disambiguated from the text.
- The abstract is truncated mid-sentence in the source ('breaks the scaling collapse barrier and'), but the body text supplies the missing claims.
- Whether Table 2's ablation uses the 6k main configuration is inferred from matching numbers, not stated explicitly.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.