Stable-MM-R1: Anchoring Multimodal Reasoning Dynamics via Entropy-Guided Stratification
Yimeng Ye, Shuang Chen, Wenxuan Huang, Manyuan Zhang, Kaituo Feng, Zhangquan Chen, Jiayu Chen, Yucheng Zhou, et al.
cs.LG · cs.CV
Quick verdict
Claims entropy-guided data curation (PAQM+HSR) stabilizes multimodal RL; sound ablations but single-run, no error bars, proprietary hints cap confidence.
Summary
This method paper targets instability and entropy collapse in reinforcement learning with verifiable rewards for multimodal reasoning. Stable-MM-R1 combines two data-centric components on top of GRPO: Potential-Aware Query Mining (PAQM), which classifies prompts by empirical success rate over N=16 pilot rollouts (keeping the 0<p<1 'Distillation Zone', downsampling solved queries, and injecting offline Seed-1.5-VL teacher hints into 'impossible' ones, reclaiming roughly 30%); and Hybrid Stratified Replay (HSR), which stratifies each step's rollouts by outcome reward and batch-percentile Path Entropy into stability-anchor, hard-negative, and epiphany buffers, then assembles groups half structured (within-stream 50% epiphanies, 25% anchors, 25% hard negatives) and half natural, clearing buffers after every step to stay on-policy. Training Qwen2.5-VL-3B/7B on a ~189k prompt corpus (Geometry3K, MM-Eureka, ViRL39K) with EasyR1 on 8×H100, the full method averages 56.2% across ten benchmarks, +9.7 over the best evaluated open 7B (Vision-G1, 46.5%); ablations attribute +3.1 to PAQM and +2.0 to HSR on top of PAQM. Main limitations: no seeds, error bars, or significance tests; baselines reported at pre-collapse peaks; a proprietary, unreleased teacher hint cache; 'Pass@8' actually means majority voting; and no code release statement, limiting reproducibility despite a fully specified algorithm.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Reward-degenerate groups (all rollouts same outcome) are a material source of vanishing updates in GRPO, and structured group composition conditionally eliminates this failure mode (§3.3.1, A.3).
- PAQM's Distillation-Zone curriculum improves the three-benchmark average by +3.1 over a GRPO baseline (Table 2: 60.7→63.8).
- HSR adds a further +2.0 on top of PAQM (65.8) and +4.0 alone over GRPO, without modifying the RL objective (Table 2).
- The full method reaches 56.2% average across ten benchmarks at 7B scale, +9.7 over the best evaluated open-source 7B (Vision-G1, 46.5%), while holding Path Entropy within roughly 0.35–0.60 nats where GRPO and DAPO collapse in their runs (Table 1, Fig. 2).
Strengths
Clean component isolation in the ablation
Table 2 runs '+PAQM' (standard GRPO objective on the identical hint-augmented curriculum), so the Full-vs-'+'+PAQM' delta of +2.0 isolates HSR under the same data pipeline; '+HSR' alone scores 64.7 vs GRPO 60.7 (+4.0), and §4.1 explicitly discloses that data-preparation costs are not matched.
Unusually honest scoping of claims
Path Entropy is explicitly disclaimed as not a Shannon-entropy or epistemic-uncertainty estimate (§3.1); the A.3 variance guarantee is stated as conditional on both reward classes being present, with natural-sampling fallback; Figure 2 commentary disclaims monotonic improvement; §4.1 admits semantic contamination cannot be fully ruled out.
Fully specified algorithm enabling reimplementation
Broad, decontaminated evaluation at two scales
Weaknesses
No uncertainty quantification anywhere
Table 1 cross-model comparison is confounded
Numerical inconsistency across tables
Nonstandard metric naming
Key motivating evidence is thin and mechanistic novelty is under-tested
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
Stable-MM-R1 (arXiv 2609.07148) is a method paper on stabilizing GRPO-style RL post-training of multimodal LLMs. It attributes instability to 'stochastic group degeneracy' — groups whose rollouts all receive the same reward yield zero group-relative advantages — and to uncurated prompt difficulty. The fix is data-centric rather than objective-level: PAQM curates prompts by empirical success rate over 16 pilot rollouts (keeping the 0<p̂<1 'Distillation Zone', downsampling solved queries, injecting offline Seed-1.5-VL hints into unsolvable ones, 'reclaiming roughly 30%'), and HSR reorganizes each step's rollouts by reward and batch-percentile Path Entropy (10th/90th) into anchor/hard-negative/epiphany buffers, assembling groups half structured (within-stream 50/25/25) and half natural, with buffers cleared every step. Central claims: PAQM +3.1 average over GRPO, HSR +2.0 more (Table 2: 60.7→63.8→65.8); 56.2% average on ten benchmarks at 7B vs 46.5% for the best evaluated open 7B (Table 1); entropy held near 0.35–0.60 nats where GRPO/DAPO collapse (Fig. 2).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is 'Rollout Silencing' in RL for LLM reasoning?
A failure mode (named in Shuffle-R1, which this paper cites) where a rollout's learning signal is drowned out because its group contains mostly identical rewards, making group-relative advantages vanish or become pure noise. Stable-MM-R1 attacks it by structuring optimization groups to always include both successful and failed rollouts when such samples exist in the current step.
Is Stable-MM-R1's 'Pass@8' the standard pass@8 metric?
No. The paper defines Pass@8 as majority-vote accuracy over eight independently sampled responses — closer to what other papers call maj@8. Standard pass@k counts success if any one of k samples is correct. The two metrics produce very different numbers, so Stable-MM-R1's scores are not directly comparable to papers using standard pass@8 or greedy decoding unless everything is re-evaluated identically.
Does Stable-MM-R1 use SFT cold start or knowledge distillation?
No supervised parameter updates are used; training is RL-only on top of Qwen2.5-VL. However, its PAQM stage injects offline teacher hints (from the proprietary Seed-1.5-VL model) into the prompts of queries the model initially fails — so teacher knowledge does enter through prompt content, just not through weight updates.
What results does Stable-MM-R1 report?
The 7B model averages 56.2% across ten multimodal benchmarks (e.g., MathVerse 56.3, MathVista 75.5, MMMU 67.2), reported as +9.7 over the best evaluated open-source 7B (Vision-G1, 46.5%); a 3B variant averages 46.2%. On a three-benchmark ablation average, GRPO scores 60.7, adding PAQM 63.8, adding HSR 64.7, and the full method 65.8.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–3 appear only as captions in the extracted text; the stability claims (entropy bands, collapse timing, trajectory shapes) rest on plotted curves whose underlying values could not be independently verified.
- Tables were extracted as flattened text; cell values were cross-checked against the printed row/column averages (all Table 1 and Table 2 averages recompute correctly), but individual cell misalignment cannot be fully excluded.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.