SE-GoS: Self-Evolving Graph-of-Skills for Skill Library at Scale
Dawei Fu, Cheng Jiang, Sitian Qian, Huainan Wang, Zhongkai Hao
cs.AI · cs.CL
Quick verdict
Proposes training-free evolution of a skill-retrieval graph from execution traces; well-calibrated but gains hover at the edge of its own ±6–7pt noise band.
Summary
This method paper introduces SE-GoS, a training-free framework that lets a static Graph-of-Skills (GoS) retrieval graph improve itself from traces an agent evaluation already produces. Three offline updates act on the retrieval substrate: topology evolution induces workflow, dependency, and avoid edges from retrieval-to-use co-occurrence (deliberately not GoS's 'alternative' relation, which observational traces cannot certify) and soft-prunes never-used heads; edge-weight evolution additively reinforces edges into skills actually used on rewarded trials (η=0.1); and a single ProTeGi-style textual-gradient pass rewrites retrieval-facing descriptions of used-but-low-ranked skills. On SkillsBench (87 tasks, 1,000-skill library, deepseek-v4-flash-0731, two attempts per task, k=1 token-overlap substrate with lexical-only seeding), one round lifts reward from 52.4% to 59.4% (+7.0) at 3.45M input tokens, above a measured SkillDAG (55.3%), and the graph transfers to a disjoint 37-task split (+5.4). A 2³ factorial shows the edge update is dominant (+4.9 alone). Multi-round evolution plateaus at round 2 and degrades by round 3 (54.0%). The paper's own ±6–7-point noise band leaves single-lever and held-out gains statistically unresolved; two of three backbones are compared against baselines quoted from the GoS paper under a different substrate; and no code release is stated.
Review complete
Assessment at a glance
6
risk flags identified
● high 0
● medium 3
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- One offline evolution round over execution traces lifts SkillsBench reward from 52.4% to 59.4% over static GoS (deepseek-v4-flash-0731, n=174 attempts) while using about 32% fewer input tokens than full skill loading.
- The evolved graph transfers to a disjoint 37-task held-out split with a +5.4-point gain, which the authors themselves place inside their ±6–7-point noise band.
- Because PPR reads edge weights directly, re-wiring and re-weighting the graph changes retrieval behavior without training parameters, editing skill content, or changing retrieval code.
- Edge-weight reinforcement is the dominant lever (+4.9 alone; leave-one-out −3.6), with topology and description updates contributing within the noise band.
- Repeated evolution plateaus at round 2 (59.8%) and degrades by round 3 (54.0%), so evolution should be applied once at deployment.
Strengths
Self-calibrated noise floor
§5.1 derives a paired SE (3–3.6 points over 87 task means) and reads all comparisons against a ±6–7-point band, explicitly flagging that the held-out +5.4 (§5.3) and single-lever ablation gains (B.1, +1.2 to +4.9) sit inside it — rare statistical candor.
Clean attribution design
A deterministic k=1 token-overlap substrate (863 semantic edges) with lexical-only seeding means evolved cells differ from static only in the applied updates; the 2³ factorial (Table 6) plus LLM-prior-free relation induction (§4.2, A.7) attribute gains to evolution rather than to a model prior.
Memorization is measured, not assumed away
Benchmark fidelity audit
Transparent nulls and unused machinery
Weaknesses
Headline sits at the edge of the paper's own band
Mixed measured/quoted baselines in Table 2
SkillDAG comparator is a re-run, not validated
Three-lever framing oversells components inside noise
No code or artifact release stated
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is SE-GoS?
A training-free method that improves a static skill-retrieval graph (Graph-of-Skills) for LLM agents by mining execution traces: it adds/prunes edges from co-occurrence evidence, reinforces edge weights into skills actually used on successful tasks, and rewrites a few skill descriptions with a single textual-gradient pass — no model training and no changes to the retrieval code.
How much does SE-GoS improve performance?
On SkillsBench (87 tasks, deepseek-v4-flash-0731, two attempts per task) one evolution round raises average reward from 52.4% to 59.4% while cutting input tokens by about a third versus loading the whole 1,000-skill library. On a disjoint 37-task held-out split the gain is +5.4 points, which the authors note sits inside their own ±6–7-point noise band.
Does evolving the graph repeatedly keep helping?
No. Reward rises to 59.4% after one round, is flat at round 2 (59.8%), and drops to 54.0% at round 3 as induced edges accumulate (863→1,502), so the authors recommend a single evolution round at deployment.
Which update component matters most?
Edge-weight reinforcement: alone it gains +4.9 points over static GoS (57.3% vs 52.4%) in the 2³ factorial, while topology (+1.7) and description updates (+1.2) fall within the paper's noise band; leave-one-out removal of edge weights costs 3.6 points.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The metadata abstract is clipped mid-sentence ('...with a 5.'), but the complete sentence (5.4-point held-out gain) is present in the full body text; no truncation found anywhere in the provided extraction (references and Appendices A-C end cleanly).
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.