Terminal-Universe: Turning Agent Trajectories into Scalable Terminal Environments
Jie Wu, Zhenru Zhang, Beichen Zhang, Xuwu Wang, Yuhui Su, Mouxiang Chen, Peng Wang, Zhihai Wang, et al.
cs.AI · cs.CL
Quick verdict
Turns agent trajectories into training environments; ablations support +11.9 TB2.1 gains, but teacher-authored verifiers and no code release warrant caution.
Summary
The paper introduces Terminal-Universe, a framework that converts existing agent trajectories into reusable, executable terminal environments for post-training data. Observing that tool-execution histories expose workspace contents, it deterministically replays read/write/edit operations to restore each file's pre-agent state, then uses an agentic completion stage to supply missing files and dependencies; a read-only judge filters task-sufficient workspaces. From 359,593 public trajectories the pipeline yields 68,263 reconstructions and 37,273 task-sufficient environments. Four re-querying variants exploit each workspace: Intent Recovery (original tasks), Single-WS (novel tasks), Cross-WS (tasks spanning two codebases linked by mined dependency relations), and Multi-Round (user-agent follow-ups with round-level hidden verification). Every task is paired with a teacher-authored pytest verifier (with a mandatory red-check), and only passing Qwen3.7-Max teacher trajectories are kept, giving roughly 32.0k SFT records (~1.42B tokens). SFT on Qwen3.5-27B lifts Terminal-Bench 2.1 from 46.2 to 58.1 (+11.9, Terminus2-XML) and EvoCode-Bench v2 MT@4 from 6.3 to 20.1 (+13.8). Ablations show re-solving beats source-trajectory SFT (52.1 vs 36.7 average), completion adds +4.2, and environment expansion dominates query/solution scaling at fixed budget. Limitations: a single teacher authors tasks, solutions, and verifiers (circularity risk); standard Ubuntu containers reduce fidelity; source-trajectory coverage bounds domain diversity; no code or data release is stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Recorded agent trajectories can be inverted into reusable executable environments via deterministic replay plus agentic completion, yielding 37,273 task-sufficient environments from 359,593 public trajectories without the original repositories.
- Re-solving recovered tasks with a consistent teacher far outperforms SFT on the original source trajectories (52.1 vs 36.7 average on Terminal-Bench 2.1, Table 4, both at 35.8k records).
- Cross-workspace (breadth) and multi-round (depth) expansions add measurable gains beyond single-workspace synthesis (+2.0 TB2.1 when added to Single-WS; +2.6 MT@4 and +5.0 case score on EvoCode v2).
- SFT of Qwen3.5-27B on the 32.0k verifier-filtered mixture improves Terminal-Bench 2.1 by +11.9 (46.2→58.1, Terminus2-XML) and EvoCode-Bench v2 MT@4 by +13.8 (6.3→20.1), with consistent Claude Code gains (+10.4).
- At fixed data budget, adding new environments beats adding queries or solutions per environment (56.0 vs 53.8/53.9, Table 10).
Strengths
Novel inversion of the trajectory-environment relationship
Prior work rolls out trajectories from environments; §1/Fig 1 recovers environments from trajectories by replaying recorded Read/Write/Edit operations, unlocking a resource (359,593 public trajectories) that prior pipelines discard — without needing original repositories.
Unusually thorough component-level ablations
§6.1–6.7 isolate every design choice with matched budgets: re-solving vs source-trajectory SFT (Table 4: 52.1 vs 36.7 TB2.1 average), completion (+4.2, Table 5), verifier filtering (Table 6), breadth/depth contributions (Tables 7/9), budget allocation (Table 10), and SWE→terminal transfer (Table 11: 47.0→50.0).
Verification discipline throughout the pipeline
Honest self-auditing and leakage hygiene
Headline gains are scaffold-consistent
Weaknesses
Single-teacher circularity with no independent audit
Weak same-base baseline coverage in Table 3
Statistical reporting lacks formal inference
Heavy single-source and language skew
No artifact release stated
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
Terminal-Universe (Qwen Team, Alibaba + Tsinghua; arXiv 2609.04148) attacks environment scarcity for terminal-agent post-training. Its core inversion claim (§1, Fig 1): a trajectory and an environment are two views of one episode, so the file-tool histories inside 359,593 public trajectories can be replayed into executable workspaces — no original repository required. Three headline claims: (1) two-stage reconstruction yields 68,263 environments, of which 37,273 pass a task-grounded sufficiency judge (Fig 7); (2) re-solving recovered tasks beats imitating the source trajectories (Table 4: 52.1 vs 36.7 average on Terminal-Bench 2.1, both at 35.8k records, verifier filtering excluded for fairness); (3) SFT on the 32.0k verifier-filtered mixture lifts Qwen3.5-27B by +11.9 on Terminal-Bench 2.1 (46.2→58.1, Terminus2-XML) and +13.8 MT@4 on EvoCode-Bench v2 (6.3→20.1). All claims are scoped to SFT; RL is invoked as motivation but never tested.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Terminal-Universe?
A framework from the Qwen team (Alibaba) that converts existing LLM agent trajectories into reusable, executable terminal coding environments. It replays the file operations recorded in a trajectory to restore the workspace state before the agent's changes, uses a completion agent to fill in missing files, and then generates new verifiable training tasks inside each reconstructed environment.
How much does Terminal-Universe improve model performance?
Per the paper's Table 3, supervised fine-tuning of Qwen3.5-27B on its ~32k-record corpus raises Terminal-Bench 2.1 from 46.2 to 58.1 (+11.9 points with the Terminus2-XML scaffold, +10.4 with Claude Code) and EvoCode-Bench v2 MT@4 from 6.3 to 20.1 (+13.8 points).
How does it reconstruct environments from trajectories?
In two stages: deterministic replay restores each accessed file to its earliest observed (pre-agent) version, yielding a partial workspace; then an agentic completion agent creates missing files and dependencies under an explicit instruction not to solve the task. A read-only judge then keeps only workspaces it deems sufficient for the recovered task.
Is the Terminal-Universe code or dataset publicly available?
The reviewed paper text does not state a public release of its own code, environments, or corpus. It links to external benchmarks (Terminal-Bench, EvoCode-Bench v2) and prints its generation prompts in appendices, but no repository for the 37.3k reconstructed environments or pipeline artifacts is given.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 3-6 numeric labels were extracted as interleaved raw text runs; every cited value was cross-checked against surrounding prose and tables, but figure-internal pairings (e.g., bar-to-label assignment) rely on label order.
- No truncation detected: the extraction runs continuously from the title through Appendix G and the full reference list.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.