APEx: Distillation of Agent Procedural Experience for Adaptive Deep Research Question Answering
Jie Ding, Rui Sun, Xinyuan Zhang, Zeyu Zhang, Xin Liu
cs.AI · cs.CL
Quick verdict
APEX distills agent experience into skills plus test-time RL for deep-research QA; SOTA claim credible, but 14.7-pt GPT-5.4 gap cherry-picks a weak baseline.
Summary
APEX is a framework for deep research question answering that converts accumulated agent interaction history into reusable procedural knowledge. Tool-use trajectories are stored as instance-level memories bucketed by task category and modality, and a Distiller compresses each bucket into an evolving category-level skill document (procedure steps, pitfalls, win rate). Executor, Distiller, and Planner are trained via three-stage alternating GRPO: the Executor learns multi-turn tool interaction, the Distiller learns reward-guided skill synthesis rather than fixed-prompt summarization, and the Planner learns plan-execute-replan decisions. At test time, retrieved skills serve as priors for skill-guided TTRL, where the Planner is updated online with multi-judge no-ground-truth rewards and confidence-weighted skill-alignment regularization. Built on Qwen2.5-VL-7B (Executor) and Qwen3-8B (Distiller/Planner) and trained on FVQA-train and MATPO, APEX averages 64.4 across seven benchmarks (FVQA-test 68.7, HotpotQA 67.8, 2Wiki 75.2, SimpleVQA 66.3, LiveVQA 42.4, InfoSeek 66.9, MMSearch 63.8), exceeding naive GPT-5.4 by 14.7 points, Gemini-3-Flash by 2.2, and the strongest memory baseline MIA by 3.0. A skill-only variant drops 1.8 points yet cuts injected memory tokens by 45.6% versus MIA. Main limitations: single-run results without significance testing, LLM-judge rewards throughout (including at test time), test-set TTRL adaptation unavailable to baselines, and figure-only evidence for token-efficiency and training analyses.
Review complete
Assessment at a glance
6
risk flags identified
● high 1
● medium 4
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Organizing experience into instance-level memories plus category-level skills and retrieving both beats the strongest memory-augmented baseline (MIA) by 3.0 average points across 7 benchmarks (64.4 vs 61.4).
- Three-stage alternating GRPO makes skill distillation reward-guided: removing GRPO from the Executor, Distiller, or Planner costs 8.4, 2.3, and 1.8 average points respectively (Table 3).
- Skill-guided TTRL enables ground-truth-free online self-improvement of the Planner worth 4.2 average points, with confidence-weighted skill-alignment regularization preventing policy drift (Eq. 16).
- A skill-only variant retains most accuracy (67.7 vs 69.5 on the Table 2 average, still 1.5 above MIA) while cutting injected memory tokens by 45.6% versus MIA.
- A 7B/8B open-source stack reaches 64.4 average, exceeding naive-prompted GPT-5.4 (49.7) by 14.7 points and Gemini-3-Flash (62.2) by 2.2.
Strengths
Reward-guided skill distillation
The Distiller is trained with GRPO on quality/format/evolution rewards (Eq. 11, mu=0.5/0.1/0.4) instead of fixed-prompt summarization; removing Distiller GRPO costs 2.3 average points (Table 3), directly supporting the core design claim.
Thorough component ablations
Table 3 isolates experience sources (w/o Both -10.1 on the 69.5 four-dataset average), per-module GRPO (w/o Executor -8.4 vs w/o Planner -1.8), and closed-loop adaptation (w/o TTRL -4.2, single-shot distillation -2.5).
Context efficiency
Above-average reproducibility disclosure
Plug-and-play executor generalization
Weaknesses
No statistical rigor
Asymmetric headline comparison
Judge circularity with unstated scoring protocol
Test-set adaptation asymmetry
Figure-only quantitative evidence
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
APEX: Distillation of Agent Procedural Experience - Preliminary Review
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is APEX?
APEX is a framework for deep research question answering in which an AI agent stores its past tool-use trajectories as instance-level memories, distills groups of them into category-level skill documents via a reward-trained Distiller module, and adapts its Planner at test time with reinforcement learning that requires no ground-truth answers.
How much better is APEX than GPT-5?
Across seven QA benchmarks APEX averages 64.4 versus 49.7 for naive-prompted GPT-5.4 (+14.7 points). The strongest frontier baseline in the comparison, Gemini-3-Flash, scores 62.2, so the margin over the best closed model is 2.2 points.
What models power APEX and can small open-source models really compete?
The Executor is Qwen2.5-VL-7B, the Distiller and Planner are Qwen3-8B, and a Qwen3-32B model judges rewards; training uses GRPO on a single 8xA100 node. The authors argue that skills plus memory let the 7B backbone rival closed models (+46.8 points over the raw Qwen2.5-VL-7B average).
Is APEX code available?
The paper links a GitHub repository (github.com/J-Ding519/APEx) and publishes all hyperparameters and prompts, but full reproduction requires 8xA100-scale RL training, live web search access, and a 32B judge model.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 3-5 are plots whose underlying per-configuration values are not recoverable from the extraction (the embedded Fig. 4 numbers are garbled); figure-based claims here rely on the prose values quoted in the text.
- The linked GitHub repository was not inspected (no network access); code availability reflects the paper's stated claim only.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.