ReDraft, Don't Just Distill: Reference-Driven Revision for Continual VLLM Post-Training
Zhihao Zhang, Mingqi Wu, Qiaole Dong, Enyu Zhou, Shuo Li, Boyang Liu, Jiazheng Zhang, Honglin Guo, et al.
cs.AI
Quick verdict
VLM cold-start post-training via self-revised verified targets; strong results and analysis, but headline metrics slightly clash with its own Table 4.
Summary
The paper targets continual post-training of vision-language models on tasks where the base model has near-zero accuracy, a regime where SFT learns quickly but forgets prior capabilities, while on-policy methods (RLVR, on-policy self-distillation) preserve behavior but supply no usable learning signal. ReDraft samples a rollout per prompt; correct rollouts are kept, incorrect ones are revised once by the same model using an expert response as reference, revisions must pass a verifier, and the model is fine-tuned on the retained set. A trust-region derivation frames the ideal target as the policy conditioned on verifier acceptance, showing SFT drops the policy weighting while self-distillation drops the hard correctness constraint; ReDraft only replaces the proposal distribution with a draft-and-revise process. On Qwen2.5-VL-3B/7B across Counting, Jigsaw, and Clock Reading, ReDraft averages +56.9 target points versus SFT's +52.9, with prior-task loss of 1.5 versus 16.6, and beats OPSD on both axes (+19.3 gain, 6.2 loss); extended-budget OPSD only reaches +23.4. Data- and weight-space analyses support the proximity claims. Limitations: reliance on a verifier and on the correction-competence assumption, stale target sets, SDFT and KL-regularized SFT baselines untested, no seeds or error bars, no stated code/data release, and the headline 56.9/1.5 figures do not exactly match the means derivable from Table 4 (56.0/2.4).
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 4
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- ReDraft achieves SFT-level or better target-task acquisition (mean +56.9 vs +52.9 points) while cutting prior-task forgetting from 16.6 to 1.5 points (claimed 11.3x).
- ReDraft dominates OPSD on both axes (+19.3 gain / 6.2 loss), and extended-budget OPSD training (+23.4) does not close the gap, indicating an objective-limited rather than optimization-limited failure.
- Verified self-revisions are more policy-proximal than expert SFT targets (median answer-only PPL 1.62 vs 5.33 at 3B) and induce parameter updates closer to SFT's direction (cosine +0.272 vs +0.135) while remaining more compact.
- The ideal verifier-conditioned target (Eq. 5) explains existing methods' failures: SFT over-weights the reference by Z/pi(y*), and self-distillation's soft tilt lacks the hard correctness indicator, so its token credit misses task-deciding positions (Figure 10).
Strengths
Principled unifying derivation
Section 3.3 and Appendix A derive the ideal verifier-conditioned target (Eq. 5) and use it to localize each baseline's exact deviation: SFT fits the point mass at the expert response and over-weights it by Z/pi(y*), self-distillation replaces the hard indicator with the unverified tilt of Eq. 13. This turns folk intuitions into precise, checkable claims.
Budget-controlled and protocol-controlled comparisons
One GPT-5.5 response set serves as SFT target, ReDraft reference, and OPSD conditioning, holding supervision quality fixed; OPSD is extended to 762 vs 449 steps (Table 2: +19.3 to +23.4, still 33.5 points behind ReDraft); GRPO's format-vs-outcome reward decomposition (Figure 11) shows RLVR fails at cold start for lack of positive rollouts, not under-training.
Multi-level mechanistic evidence
Transparent rates and honest limitations
Weaknesses
Internal numeric inconsistency in headline claims
Ground-truth leakage in the revision stage
Missing regularization baseline
No variance reporting; unstated checkpoint selection
Reproducibility gaps
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
ReDraft addresses continual post-training of vision-language models in the cold-start regime: adding a capability the base model cannot yet perform (single-rollout base accuracy 0.03%/0.21% at 3B on Clock/Jigsaw, per Tables 3 and 5) without eroding prior capabilities. Claimed contributions: (1) a method converting the policy's own failed rollouts into verifier-approved self-revised SFT targets; (2) SFT-level acquisition (+56.9 mean target points vs +52.9) with 11.3x less forgetting (1.5 vs 16.6 prior-task points); (3) mechanistic attribution to policy proximity in data space and SFT-direction alignment in parameter space. The theoretical frame (Section 3.3, Appendix A) derives the ideal target as the current policy conditioned on verifier acceptance (Eq. 5) and diagnoses each family precisely: SFT fits the point mass at the expert response, over-weighting it by Z/pi(y*); self-distillation replaces the hard indicator with the unverified tilt of Eq. 13; ReDraft keeps the indicator while substituting the draft-and-revise proposal rho (Eqs. 7-8). The derivation is correct and genuinely diagnostic; its practical force rests on how close rho stays to the evolving policy, which the paper measures under the frozen base model rather than bounds.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is ReDraft?
ReDraft is a post-training method for vision-language models that turns the model's own incorrect answers into training data: it samples an answer, has the model revise its mistakes using an expert answer as a reference, keeps only revisions that a verifier accepts, and fine-tunes on the surviving set. The goal is SFT-like learning speed on new tasks with far less forgetting of old capabilities.
How much better is ReDraft than SFT?
Averaged over Counting, Jigsaw, and Clock Reading on Qwen2.5-VL-3B/7B, ReDraft gains about 57 points on the new task versus SFT's 53, while losing only about 1.5 points on eight prior-capability benchmarks versus SFT's 16.6 - roughly 11x less forgetting. Note: the per-run appendix table averages to +56.0/-2.4 for ReDraft, slightly different from the headline figures.
Does ReDraft need ground-truth answers?
It needs (a) an expert reference response per prompt, used as context for revision, and (b) an automatic verifier that judges whether a complete response is correct. Notably, the revision prompts shown in the paper's appendix include the correct answer directly, which a critical reader may view as answer injection rather than pure self-correction.
Is code or data available for ReDraft?
The paper text does not mention a public code or data release. The expert training targets were generated with the proprietary GPT-5.5 model, so exact reproduction would require regenerating comparable targets; the underlying task corpora (PixMo-Count, COCO, synthetic analog clocks) and base models (Qwen2.5-VL-3B/7B) are public.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1 and 4-15 are text-extracted; exact plotted values are partially reconstructed from axis labels and bracket annotations, so figure-vs-table discrepancies flagged here may partly reflect extraction artifacts.
- Appendix B.3's narrative is truncated mid-sentence before Table 4, but the table itself appears intact.
- Code/data availability was assessed only from the paper text; no supplementary links were extractable.
- Citations include 2026-dated works that could not be verified offline.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.