SkillAdam: Stable and Efficient Skill Evolution for Agents
Gaoyuan Li, Meihao Fan, Yizhe Liu, Shaolei Zhang, Ju Fan, Siyi Wang, Jiaheng Hou, Xudong Weng, et al.
cs.AI
Quick verdict
Claims Adam-inspired memory+budget stabilize skill self-evolution; broad gains over SkillOpt are plausible but small and lack significance testing.
Summary
The paper frames iterative optimization of natural-language agent skill documents as a discrete analog of Adam. SKILLADAM maintains two persistent optimizer states: an Evolving Issue Tracker that records identified problems, their status, and prior solution attempts (a functional first moment), and a volatility-driven edit budget that exponentially weights the variance of case-level improvements to scale the next edit's scope (a functional second moment). A trajectory-informed initialization and a multi-metric acceptance gate complete the loop. On five short-horizon benchmarks (using GPT-5.5 baselines taken from the SkillOpt paper), SKILLADAM edges out SkillOpt by 0.2-1.1 percentage points, with one tie on OfficeQA; on long-horizon DeepPlanning (Claude Sonnet 4.5), it raises DP-Avg from 21.7% to 28.3% while cutting total optimization tokens by 67.3%. A cumulative ablation on DeepPlanning attributes 6.7 points to the edit budget and 2.5 to memory; transferred skills retain more performance on GPT-5.4-mini. Main limitations: no significance tests or multi-seed variance despite case-level margins on 60-case splits (DP-Travel 11.7% vs 1.7% is 7 vs 1 cases), single-run dynamics analysis, cumulative-only ablation (the no-memory variant even wins Shopping L1), and unreported hyperparameters. A code repository is stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- An Evolving Issue Tracker recording problems, statuses, and prior fix outcomes acts as a functional Adam first moment that stabilizes the update direction of discrete skill optimization.
- A volatility-driven edit budget, an EMA of case-level improvement variance, acts as a functional second moment that adapts each revision's scope to the consistency of recent gains.
- SKILLADAM achieves state-of-the-art across seven benchmarks, edging SkillOpt on short-horizon tasks (0.2–1.1 points, one tie) and raising DeepPlanning DP-Avg from 21.7% to 28.3%.
- It reaches stronger skills at 67.3% lower total token cost and 68.8% fewer API requests than SkillOpt on DeepPlanning.
- Skills optimized by SKILLADAM transfer to GPT-5.4-mini with higher retention than SkillOpt's (81.3% vs 76.2% average).
Strengths
Explicit, honest functional framing
Table 1 makes the Adam correspondence item-by-item, and the text repeatedly states the analogy is functional rather than numerical (§4.2) and flags its own limits: cumulative ablation (§5.5), judge experiment (§5.7), single-run dynamics (§6.1).
Paired evaluation design
The candidate skill is evaluated on the same mini-batch as the current skill (Eq. 22), removing a classic confound in iterative prompt/skill optimization; the multi-metric acceptance gate (§5.3) with protected-metric regression bounds is a sensible safeguard.
Large long-horizon gains where baselines collapse
Coherent efficiency evidence
Real transfer experiment
Weaknesses
No statistical inference anywhere
Cumulative-only ablation with an unexplained reversal
Imported baseline numbers for five of seven benchmarks
Key hyperparameters and gate thresholds never specified
Self-judging skill quality
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
SKILLADAM (arXiv 2609.08944, Renmin University + Tencent) targets iterative optimization of a single natural-language skill document for a frozen LLM agent. The authors identify two failure modes of prior skill self-evolution — Direction Stability (iteration-local feedback overwrites effective corrections) and Update Adaptivity (edit scope ignores how consistent recent gains are) — and map two optimizer states functionally onto Adam (Table 1): an Evolving Issue Tracker recording issues, their status, and prior fix attempts with outcomes, as first-moment analogue; and a volatility-driven edit budget sigma_t+1 = max(bmin, bbase(1 - clip(Vt/Vmax, 0, 1))), with Vt an EMA (coefficient beta2) of the per-case score-delta variance, as second-moment/step-size analogue. Headline claims: state-of-the-art on seven benchmarks, more stable dynamics, fewer iterations, lower cost, better cross-model transfer. Credit where due: the paper repeatedly and explicitly states the analogy is functional, not numerical (§4.2), and flags the limits of its own cumulative ablation (§5.5), judge experiment (§5.7), and single-run dynamics analysis (§6.1).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is SkillAdam?
SkillAdam is a framework from a 2026 arXiv paper (Renmin University and Tencent) that iteratively improves natural-language 'agent skill' documents for LLM agents. It borrows two ideas from the Adam optimizer: an issue tracker that remembers past problems and attempted fixes (analogous to Adam's first moment), and a volatility-based edit budget that controls how much each revision may change based on how consistently recent changes helped (analogous to Adam's second moment).
How much better is SkillAdam than prior methods?
Per the paper's single-run experiments, it slightly beats the closest prior method SkillOpt on most benchmarks (0.2–1.1 percentage points on five short-horizon tasks, one tie) and shows larger gains on the long-horizon DeepPlanning benchmark (28.3% vs 21.7% average accuracy) while using about 67% fewer optimization tokens there. No statistical significance tests or multi-seed variances are reported, so the small short-horizon margins should be read cautiously.
Is code available for SkillAdam?
The paper states a GitHub repository (github.com/ruc-datalab/SkillAdam). However, the paper text does not report the hyperparameter values (edit budget parameters, volatility threshold, acceptance thresholds), so exact replication from the paper alone is not currently possible.
Which models and benchmarks were used?
GPT-5.5 served as the frozen target agent on six benchmarks (SearchQA, SpreadsheetBench, OfficeQA, DocVQA, LiveMathematicianBench, ALFWorld) and Claude Sonnet 4.5 on DeepPlanning shopping/travel planning. Skills optimized with GPT-5.5 were also tested unchanged on GPT-5.4-mini to measure cross-model transfer.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 3 and 4 exist in the PDF as images; only their captions, axis labels, and the aggregate numbers stated in the body text (e.g., mean judge score 3.30 to 3.88) are recoverable from the extraction, so per-slice judge scores and per-iteration dynamics values could not be verified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.