One to More, More to One: Category-Aware Iterative Expert Training for Software Engineering Agents
Jie Zhao, Ziyu Jiang, Suhang Zheng, Minghui Shan, Xiaoxiao Xu, Lin Qu
cs.SE · cs.CL · cs.LG
Quick verdict
Alibaba trains three category RL experts for SWE agents and merges them via multi-teacher distillation; gains look real but a compute-matched single-policy baseline is missing.
Summary
The paper addresses uneven, category-level progress during pooled reinforcement learning of repository-level SWE agents, where aggregate resolution can hide regressions in some task groups. Using Pro-618, an audited 618-instance subset of SWE-bench Pro, the authors build SWE Labeler, a hierarchical taxonomy grounded in ISO/IEC 25010, CWE, and Fowler; a Refresh-Repair-Expand loop alternating Agentic-miniRL with SFT on the policy's own verifier-approved trajectories; and label-routed multi-teacher on-policy distillation with ReLU-gated reward extrapolation, consolidating three category experts into one model. The final policy reaches 58.04% on Pro-618 (+5.39 points over the Qwen3.6-27B base) and 59.00% on SWE-bench Multilingual (+2.78 points), exceeding Pooled RL and Balanced RL in every category. Strengths include honest process metrics (Gsim, see-saw gap), convincing instance-level Repair SFT recovery evidence, and a well-documented pipeline. Main limitations: no compute-matched single-policy baseline against the three-expert pipeline, no component ablations, reliance on proprietary Qwen3.7-max labeling, and no stated code release; Expert B's advantage over Pooled RL (2.16 points) sits near evaluation noise.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Joint RL on pooled SWE tasks exhibits a 'category see-saw': aggregate resolution gains conceal opposing per-category movements, which the paper quantifies via G_sim and the see-saw gap (SSG) on Pro-618.
- Category experts developed through iterative Refresh–Repair–Expand (Agentic-miniRL alternating with Repair SFT on the policy's own verified trajectories) reach +7.84/+4.48/+5.61 points over base on Pro-A/B/C without any external teacher model.
- Label-routed multi-teacher on-policy distillation with ReLU-gated reward extrapolation consolidates the three experts into one deployable model that improves over the base by 5.39 points on Pro-618 (58.04%) and 2.78 points on SWE-bench Multilingual (59.00%).
- The integrated MOPD policy outperforms both Pooled RL and Balanced RL overall and in each of the three categories on both benchmarks, constituting a category-wise Pareto improvement over Pooled RL.
- Expert-gain recovery in the single model is uneven across categories (full recovery for A at 58.07% vs expert 59.58%, overshoot for B at 59.37% vs 58.87%, only ~85% for C at 56.63% vs 57.48%), which the authors flag as open room for better integration.
Strengths
Motivating phenomenon is quantified, not just asserted
Section 3.2 formalizes the category see-saw with explicit metrics (G_sim as minimum per-category gain, SSG = Δ_all − G_sim) and Section 6.1/Table 8 shows Balanced RL raises mean G_sim (0.19 vs 0.08) yet fails to shrink SSG (1.18 vs 1.05) — an honest negative result that motivates the expert route.
Repair SFT effect is supported at both instance and benchmark level
Table 9 shows training success after Repair SFT of +10.23/+14.05/+9.47 points over post-RL (A/B/C); of 839 records that regressed during initial RL, 751 entered repair training and 61.3–73.0% recovered to or above base rates; Table 11 shows the corresponding Pro-618 target-category means rising monotonically across the four RRE stages.
Careful benchmark hygiene and evaluation protocol
End-to-end pipeline is unusually well documented
Weaknesses
No compute-matched single-policy baseline
Expert advantage over Pooled RL is thin exactly where it matters, and untested statistically
No component ablations
Proprietary and self-selected evaluation infrastructure
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This Alibaba Group preprint (arXiv 2609.23377, base model Qwen3.6-27B, artifact named Logics-SWE-Qwen3.6-27B) targets a real and precisely articulated problem: under pooled agentic RL on repository-level SWE tasks, aggregate resolution can rise while some task categories regress — the "category see-saw." The claimed contribution is threefold: (1) SWE Labeler, a hierarchical evidence-grounded taxonomy (26 Task Type / 21 Repository Domain L1 families, 227 L2 labels, 12 scale levels) whose Repository Domain L1 axis is deterministically mapped into three routes A/B/C; (2) Refresh–Repair–Expand (RRE), an expert-development loop alternating Agentic-miniRL with Repair SFT on the expert's own verifier-approved trajectories; (3) label-routed multi-teacher on-policy distillation (MOPD) with ReLU-gated reward extrapolation that consolidates the three experts into one deployable student. The headline claims: MOPD reaches 58.04% on Pro-618 (+5.39 over base; Table 12) and 59.00% on SWE-bench Multilingual (+2.78), beating both Pooled RL (55.50%) and Balanced RL (55.34%) in every category.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the main contribution of this paper?
It proposes organizing repository-level software engineering tasks into three categories (service/data/security, user-facing, systems/tooling), training one RL expert per category, and merging them into a single model via label-routed multi-teacher on-policy distillation. The final model scores 58.04% on an audited SWE-bench Pro subset (Pro-618) and 59.00% on SWE-bench Multilingual, improving over its Qwen3.6-27B base by 5.39 and 2.78 points.
Does the merged model beat simply training on all tasks together (Pooled RL)?
On Pro-618, the merged model scores 58.04% versus 55.50% for Pooled RL and beats it in every category. However, no baseline was run with a single policy given the same total training compute as the three-expert pipeline plus distillation, so the gain cannot be cleanly attributed to the category-aware design rather than extra training volume.
Is the code or the trained model available?
The provided text contains no code, data, or model release statement. Training depends on internal assets: a ~32K synthetic task pool, a proprietary labeling model (Qwen3.7-max), and the Logics-SWE-Qwen3.6-27B artifact, so independent reproduction would be very difficult as described.
What is Pro-618 and why not use the full SWE-bench Pro?
Pro-618 is the authors' audited subset of SWE-bench Pro (618 of 731 tasks), excluding instances flagged by public audits for specification defects, unreliable graders, or reference-patch failures, plus one OpenAI-documented prompt-test mismatch. All methods are compared on the same subset, but headline numbers are therefore not comparable to published full SWE-bench Pro results, and the authors explicitly note results do not generalize to the unexecuted full set.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The extraction is heavily truncated: the MOPD construction subsection, Multilingual per-category results (Table 13), anti-hacking controls, and most appendices are missing, so some judgments rest on the visible abstract, Section 6, and conclusion.
- Figures 1-7 are reduced to captions and axis labels; reported curve shapes (e.g., see-saw trajectories, KL decline) could not be independently verified from the text.
- Two Multilingual baseline numbers (Pooled 57.83, Balanced 57.17 vs base 56.22) appear only in a truncated fragment and could not be fully context-checked.
- arXiv IDs cited as 2026 preprints (MOPD, ExOPD, SWE-rebench V2) could not be verified against the registry per no-network constraint.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.