Dependency-Aware Trajectory Refinement for Efficient Multi-Turn Agent Fine-Tuning
Zhuo Chen, Zhuo Chen, Zhen Zhang, Xinyu Wang, Xinyu Wang, Kewei Tu
cs.CL
Quick verdict
The paper recasts agent-trajectory cleanup as deterministic editing of a round-level dependency DAG and reports accuracy and efficiency gains; promising and well-analyzed, but headline margins lack statistical support and the checkpoint-selection protocol favors the proposed methods.
Summary
The paper proposes refining multi-turn ReAct agent trajectories before SFT by representing each trajectory as a round-level dependency DAG (extracted by GPT-5.4), then applying deterministic graph edits: leaf pruning of dead-end rounds (L1), strict merging of sibling rounds with identical parents and children (L2), relaxed iterated merging (L3), plus LLM-rephrased variants (L2a/L3a). On 6,196 multimodal search trajectories from WebWatcher, fine-tuning Qwen3-VL-30B-A3B-Thinking yields +1.7 pp average accuracy over vanilla SFT (L1: 50.04% vs 48.35%) across SimpleVQA, LiveVQA, HLE, and MMSearch, while L2a cuts per-sample inference tokens from 20.9K to 11.0K (about 48%) at near-vanilla accuracy. The strongest analysis section shows why aggressive merging backfires: multi-call tool responses violate the serial one-call-per-turn inference loop, raising stuck rates 2-3x until rephrasing repairs the format gap. Main limitations: single seed with no significance tests on margins of 0.3-2.7 pp; asymmetric checkpoint selection (one checkpoint evaluated for baselines, four for the proposed methods); results confined to one model family and multimodal QA; DAG annotation and rephrasing both depend on GPT-5.4.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 4
● low 0
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Recasting each agent trajectory as a round-level dependency DAG turns trajectory refinement into deterministic, interpretable graph editing with three concrete edit levels.
- SFT on DAG-refined trajectories beats vanilla SFT by up to 1.7 pp average accuracy (L1, 50.04% vs 48.35%) across four multimodal QA benchmarks.
- Strict merge with LLM rephrasing (L2a) matches vanilla-SFT accuracy while cutting per-sample inference tokens about 48% (20.9K to 11.0K) and messages about 40% (23.94 to 14.30).
- LLM round-deletion without dependency awareness (Critical-Path) is lossy, dropping average accuracy to 44.32%, below even the zero-shot baseline.
- Merged-round format mismatch with the serial one-call-per-turn inference loop explains why un-rephrased relaxed merging (L3) degrades efficiency, and LLM rephrasing largely closes this gap.
Strengths
Transparent, deterministic refinement built on an interpretable intermediate representation
Recasting a trajectory as a round-level dependency DAG (Sec. 2.2) makes edits pure graph rules (Sec. 2.3), auditable and cheap; Sec. I walks a worked example (_idx=1581) step by step, including edge-level rationales and the safe-by-construction handling of round 1. This is materially more reproducible than LLM-judged deletion.
The format-gap analysis (Sec. 4.1, Sec. E) is a genuine contribution beyond the headline numbers
Merged rounds bundle k tool calls into one turn while the inference loop is strictly one-call-per-turn (Fig. 5); this explains why un-rephrased L3 inflates inference tokens to 31.8K vs vanilla SFT's 20.9K and raises stuck rates 2-3x (Fig. 3), and why rephrasing (L3a) repairs most of the gap. The stated design rule gives practitioners actionable guidance.
The DAG view diagnoses why the deletion baseline fails, not just that it fails
Efficiency claims are backed by per-benchmark breakdowns, not just averages
Unusually complete appendices including honest negative results
Weaknesses
The headline comparison is exposed to checkpoint-selection bias
No variance quantification anywhere
Data-composition confound is not isolated
Abstract overclaims relative to the full variant set
Annotator dependence and judge overlap are acknowledged but not stress-tested
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is dependency-aware trajectory refinement for agent fine-tuning?
It is a training-data cleaning method: before fine-tuning a tool-using LLM agent, each multi-turn trajectory is converted into a dependency graph showing which rounds' outputs are actually used by later rounds and the final answer. Rounds nothing depends on (dead ends) are deleted, and independent parallel sub-steps feeding the same next step are merged into one round. The edits are deterministic graph operations, optionally followed by an LLM rewrite that makes merged rounds read naturally.
How much does it improve accuracy and efficiency?
On four multimodal QA benchmarks averaged together, leaf pruning (L1) raises accuracy by about 1.7 percentage points over vanilla SFT (50.04% vs 48.35%). The strict-merge-with-rephrasing variant (L2a) keeps roughly vanilla accuracy while cutting per-sample inference tokens from about 20.9K to 11.0K (about 48% fewer) and messages from 23.9 to 14.3. These are single-run numbers without significance testing.
Why can simply deleting redundant rounds hurt agent training?
The paper's deletion baseline (an LLM judge marking rounds to remove) drops accuracy below even the untuned zero-shot model. Deleting a round whose facts are still cited by the final answer leaves the trained model 'hallucinating sources' it never retrieved. The dependency-graph approach avoids this by only removing rounds nothing downstream consumes, and by merging rather than deleting parallel lookups.
Which model and data were used, and is code available?
The base model is Qwen3-VL-30B-A3B-Thinking, fine-tuned on 6,196 multimodal search trajectories extended from the WebWatcher dataset. Evaluation uses SimpleVQA, LiveVQA, HLE, and MMSearch with a GPT-5-nano answer judge. The paper states that code is available (a repository link is provided in the paper).
Should practitioners use leaf pruning or merging?
The paper's evidence supports leaf pruning (L1) when accuracy is the priority and strict merging with rephrasing (L2a) when inference cost matters most. Un-rephrased relaxed merging (L3) is not recommended: it bundles multiple tool calls into single turns, conflicting with the serial one-call-per-turn format of inference loops, which raised stuck-loop rates and made it the most expensive SFT variant at inference despite the smallest training data.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Review is based on extracted text only: figures (Figs. 1-6) and merge-statistics tables appear as OCR fragments, so plot-level claims (trade-off curve shape, stuck-rate patterns) could not be independently verified.
- The abstract's 'up to ~40% fewer messages' corresponds to 23.94 vs 14.30 messages, which compares L2a against vanilla SFT; smaller-stated variants of the efficiency claim could not be cross-checked beyond Tables 2 and 5-6.
- Code availability is taken from the sentence 'Code is available here'; the linked repository could not be inspected, so actual reproducibility is unverified.
- Some appendix sections (A-B settings, per-prompt details beyond those quoted in Sec. J) are only partially present in the extraction; judgments about them rely on the quoted tables.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.