AgentBrew: Offline Tool-Use Agent Learning from Raw Real-World Trajectories
Zhiyi Lyu, Yewen Li, Longtao Zheng, Shengtian Yang, Lang Feng, Lei Feng, Peng Jiang, Kun Gai, et al.
cs.AI
Quick verdict
Offline hindsight relabeling + PMI action credits lift Qwen3-32B by +8.7 Acc avg on 3 MCP apps; promising, but small evals lack variance reporting.
Summary
AgentBrew is an offline training framework that turns a single batch of raw, unfiltered tool-use trajectories from real MCP applications (GitHub, Notion, PostgreSQL) into supervised fine-tuning signal. The agent explores the live environment, proposes state-grounded tasks, and executes them (roughly 2,500-3,500 trajectories per app; only max-turn failures are discarded). The corpus is then distilled in two steps: retrospective task inference uses an LLM to rewrite each trajectory's instruction so it matches what was actually accomplished, and PMI-based credit assignment uses the chain rule of pointwise mutual information under a reference model to decompose trajectory-level information into additive per-action credits, which are clamped, normalized to [0,2], and used to weight the SFT loss. Qwen3-32B serves as base policy, reference model, and inference model simultaneously. On mcp-universe/mcpmark evaluations (apparently only tens of tasks per app), the paper reports 18.2 average Acc / 58.7 Score, exceeding Qwen3-235B (15.9/54.3), Vanilla SFT, LLM-judge rejection sampling, and TOUCAN, though absolute accuracy remains far below Gemini-3 Pro (57.1/80.8). Ablations support both components, adding lower-quality data helps monotonically, and the distilled data transfers to a 14B model. Main limitations: no variance estimates or significance tests on the small evaluation sets, a self-referential single 32B pipeline, alignment analysis that relies on the GPT-5 judge the paper itself deems unreliable, and coverage of only three applications.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A single batch of unfiltered raw trajectories from a real application suffices to train a better tool-use agent, with no verifiers, simulators, or further environment interaction.
- Retrospective task inference, which rewrites each instruction to match the trajectory's actual outcome, is necessary: on Notion it lifts training from 7.1/65.6 to 14.2/70.1 (Table 2).
- PMI-based per-action credits outperform uniform weighting, LLM self-scoring, and InfoPO on GitHub and Notion (Fig. 3a-b) and automatically suppress misaligned inferred tasks (Fig. 3c-d).
- AgentBrew-trained Qwen3-32B reaches 18.2 average Acc / 58.7 Score across three MCP applications, above Qwen3-235B (15.9/54.3) and rejection sampling (12.3/48.4), and the distilled data transfers to a 14B model (20.2/55.3 vs 8.7/42.8 base).
Strengths
Deployment-realistic problem framing
§1–2 define a setting practitioners actually face: no pre-defined tasks, no verifiers, no simulator, one batch of interaction. Stages 2–3 are genuinely offline, which the paper correctly identifies as the practical differentiator versus on-policy RL lines (GRPO, WebRL, GiGPO).
Principled, exact credit decomposition
Eqs. 2–4 use the chain rule of PMI so per-action credits telescope to exactly the trajectory's total information about the inferred instruction (ΔL = L∅ − L_Tn); the clamp rationale (uncertainty-increasing actions repaid by the resolving step) is coherent, and credit computation is cheap via KV-cache reuse (~9 h on 4×H100 for ~8,500 trajectories, §4.1).
Well-controlled baselines and coherent ablations
Unusually complete reproducibility material
Honest absolute-level reporting
Weaknesses
No statistical uncertainty reporting on tiny evaluation sets
Self-referential single-model pipeline
Internal tension on LLM judges
Narrow evaluation surface
Unexamined train/serve mismatch from hindsight relabeling
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
AgentBrew (arXiv:2609.05837; NTU, Kuaishou, Southeast University) targets a deployment-realistic problem: training a tool-use agent for one specific application when there are no pre-defined tasks, no verifiers, no simulator, and only a single batch of environment interaction is affordable (§1–2). Central claims: (1) raw, unfiltered trajectories from real MCP applications contain recoverable supervision; (2) hindsight rewriting of instructions ('retrospective task inference', §3.2.1) makes instruction-trajectory pairs coherent; (3) per-action credits from the chain rule of pointwise mutual information (§3.2.2) beat trajectory-level filtering; (4) this lifts Qwen3-32B above Qwen3-235B on three real applications.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is AgentBrew?
AgentBrew is an offline training framework for LLM tool-use agents. It takes one batch of raw, unfiltered interaction trajectories collected on real applications (GitHub, Notion, PostgreSQL), rewrites each trajectory's instruction to match what was actually accomplished, weights individual actions by mutual-information-based credits, and fine-tunes the model with that weighted supervised loss — without task verifiers, simulators, or iterative environment interaction.
How does AgentBrew assign credit to individual actions?
For each trajectory, it measures how much each action-observation pair reduces a reference model's negative log-likelihood of the retrospectively inferred instruction. By the chain rule of pointwise mutual information, these per-action values sum exactly to the trajectory's total information about the instruction. Negative credits are clamped to zero and weights are normalized to [0,2] before being applied to the SFT loss.
What results does AgentBrew report?
On mcp-universe/mcpmark evaluations of three MCP applications, fine-tuned Qwen3-32B averages 18.2 Acc / 58.7 Score versus 9.5/49.0 for the base model, exceeding Qwen3-235B (15.9/54.3), Vanilla SFT (10.3/48.1), LLM-judge rejection sampling (12.3/48.4), and TOUCAN (11.9/44.8), though still well below Gemini-3 Pro (57.1/80.8). Gains also transfer to a 14B model (20.2/55.3 vs 8.7/42.8).
Is AgentBrew's code or data available?
The paper states code is available at github.com/alphatogo/AgentBrew, and all prompts, hyperparameters, and corpus statistics are given in the appendices. The collected trajectory corpus itself is not explicitly stated as released, and it was gathered from live application states that change over time.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Extraction is complete through page 23 (references and Appendices A–G included); no truncation was detected, and values embedded in Figures 3 and 4 were present as text labels.
- Per-environment test-set sizes are not stated in the text; the counts cited (~28 per env, ~21 for Postgres) are inferences from Acc granularity and are labeled as such.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.