SkillGLoW: Procedural-Family Skill Consolidation for Self-Improving Agents on Long-Horizon Task Streams
Ao Yan, Xin Zhang, Jiawei Du, Joey Tianyi Zhou
cs.AI
Quick verdict
Claims procedural-family granularity beats single-doc and flat-pool skill stores; evidence across 12 runs is credible but small-scale, single-trial.
Summary
The paper asks at what granularity an LLM agent should organize self-written skills. On long-horizon task streams, a single global document collapses into generic discipline while a flat per-task pool inflates and stays instance-bound. SkillGLoW (GLoW) introduces the procedural family as the unit of reuse: per-task local skills distilled from trajectory differences are clustered by how tasks are solved (multi-view embeddings plus consensus clustering), compressed into de-instantiated global priors, and admitted only through a verifier-grounded commit gate that requires the candidate library to match or beat the deployed one on real execution. Instance detail is regenerated per task rather than stored. Across four benchmarks (Terminal-Bench-Pro, SWE-bench Verified, ALFWorld, LiveMathematicianBench), three models, and 12 continual-improvement runs, frozen priors gain +17.2 points (hard) over the no-skill baseline on average (+18.0 with local regeneration), positive in all 12 runs (Wilcoxon p=0.000488); the committed library is 3.6x more compact than the per-task pool; GLoW leads SkillOpt on 15 of 21 cells; and priors transfer unmodified to unseen ALFWorld tasks (73.9% to 83.9%). Main limitations: tiny task streams (20-53 tasks), one trial per cell at temperature 0.7, best-across-rounds read-out, per-benchmark gains often within the measured 5.4 pp reproduction band, and no public code release stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- The procedural family — one shared solving procedure across a cluster of related tasks — is the missing unit of skill reuse between the single-document and per-task-pool organizations.
- Compressing family local skills into de-instantiated global priors, admitted only through real-execution non-degradation, gains +17.2 points (hard) over no-skill on average across 12 continual runs (+18.0 with local regeneration), positive in all 12.
- The committed library holds one prior per procedural family and is 3.6x more compact than the per-task pool while performing better.
- Under an aligned protocol GLoW leads the published single-document optimizer SkillOpt on 15 of 21 comparison cells.
- Frozen priors transfer unmodified to unseen tasks (ALFWorld mean hard success 73.9% to 83.9%), indicating procedure rather than task memory is what is consolidated.
Strengths
Measurement honesty unusual for this area
Reproduction dispersion is measured, not assumed: 18 repeated round-0 no-skill runs yield pooled sigma=2.71pp (Table A20); sec. J.2 derives a 5.4pp per-cell resolution band and concedes some per-benchmark gains fall inside it; gate decisions are logged with margins (19 accepts / 7 rejects; two epsilon-band admissions at -1.08/-1.81pp, sec. D.3).
Organizational ablation isolates granularity from optimizer effort
Table 2 measures the authors' own library at both extremes (Base-only mean +2.0, Local-only +10.9, hard) plus an AWM-style flat retrieval pool (+5.0) under identical conditions, while SkillOpt — official repo, internal hyperparameters untouched, neutral 99-byte initial skill verified zero-contribution in one cell (sec. C.1) — serves as the optimized single-document control.
No per-cell tuning; one hyperparameter set across all 12 runs
Transfer with an unmodified library
Weaknesses
Under-powered per-cell effects
Best-across-rounds (peak) read-out
Clustering instability and coverage gaps weaken the core narrative
Thin external baseline coverage and no stated code release
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper addresses an organizational question rather than a new learning algorithm: in what form should an LLM agent maintain the textual skills it writes from its own executions? The authors argue the two dominant forms fail in opposite ways on long-horizon streams where tasks demand different solutions — a single global document (SkillOpt-style) collapses into generic discipline, while a flat per-task pool inflates and stays bound to the instance that wrote it. The proposed missing unit is the **procedural family**: the solving procedure shared by a cluster of related tasks. GLoW consolidates per-task local skills into one de-instantiated global prior per family, regenerates instance detail at solving time, and admits priors only when real execution shows no degradation. Central claims: +17.2 points (hard) over no-skill across 12 continual runs (+18.0 with local regeneration), positive in all 12 (Wilcoxon p=0.000488, Table A8); a 3.6x more compact library than the per-task pool (Tables A9-A10); a 15/21-cell lead over SkillOpt under an aligned protocol; and unmodified transfer (ALFWorld 73.9→83.9, Table 4; SWE 40.0→45.6 on 30 held-out instances, Table A12).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is SkillGLoW?
SkillGLoW (GLoW, Global-Local Weave) is a framework for organizing the textual skills an LLM agent writes from its own executions. It groups per-task local skills into procedural families — clusters of tasks sharing a solving procedure — and compresses each family into one de-instantiated global prior. A new task receives the frozen prior plus a freshly regenerated task-specific skill, and a prior enters the long-term library only when real execution shows it does not degrade the deployed library.
How much does SkillGLoW improve agent performance?
Across four benchmarks (Terminal-Bench-Pro, SWE-bench Verified, ALFWorld, LiveMathematicianBench) and three models (DeepSeek-V4-Pro, MiniMax-M3, GPT-5.4-mini), the frozen priors improved hard success by 17.2 points on average over a no-skill baseline (18.0 with local skill regeneration), with positive gains in all 12 continual-improvement runs (Wilcoxon p=0.000488). On 60 unseen ALFWorld tasks the unmodified library raised mean success from 73.9% to 83.9%.
How is GLoW different from a single skill document or a flat skill library?
The paper argues a single document is too coarse — it merges distinct procedures into generic advice — while a per-task pool is too fine — entries overfit the instance that wrote them and the pool grows unbounded. GLoW stores one de-instantiated prior per procedural family. Its own ablation (Table 2) shows Base-only (single compressed document) averaging +2.0 points, Local-only +10.9, a flat pool with retrieval +5.0, and Global-only +17.2 on the same 12 runs.
Is code or data available for SkillGLoW?
The paper states the supplementary material ships the outputs of all twelve runs (per-round summaries, configurations, library states, committed priors, local skills), the four fixed task-split files with seeds, and 24 prompt templates. No public code repository for GLoW itself is mentioned in the provided text; the compared SkillOpt baseline uses the official microsoft/SkillOpt implementation.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The abstract supplied in the metadata cuts off mid-sentence (from 7...), but the complete result (73.9% to 83.9%) appears in sec. 4.5 and Table 4 of the embedded full text, so no result is missing.
- Figure 1/2 label text is interleaved into body prose by PDF extraction (e.g., after the SkillsBench sentence in sec. 1); all table numbers read cleanly and nothing in the reference list or appendix appears truncated.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.