Agent as Policy for Robotic Manipulation
Mengzhao Jia, Yang Lin, Xixin Zhang, Zhihan Zhang, Xiaobai Liu, Meng Jiang
cs.CL
Quick verdict
Frontier coding agents as full robot policies show strong zero-shot real-robot success (80-100% in 7/8 configs) but lack baselines and statistical rigor.
Summary
The paper introduces Agent as Policy (AGP), which places a general-purpose coding agent — not a trained policy — in direct control of a real robot arm throughout task execution. A preparation agent writes a reusable task definition; an execution agent then interprets camera/depth observations, writes Python programs for geometry and pose estimation, and issues motion and gripper commands through a calibrated robot bridge, revising actions from physical feedback with model weights frozen. On an I2RT YAM arm, AGP completes four-pair assembly from human video (8/10), three block-construction configurations from goal images (10/10, 10/10, 9/10), dice flipping (10/10), targeted throwing (2/2), and bimanual towel folding (5/5 sequential; 3/5 simultaneous) — at least 80% success in seven of eight configurations. Comparisons across agents and models (Codex, Claude Code; GPT-6 Astra, GPT-5.6 variants, Claude Opus 5) show model choice matters. Experience saved as files cuts two-pair assembly time 29.3% by the fifth execution, and transferring Astra experience to the weaker Terra model raises success from 1/5 to 4/5. Main limitations: no baseline against alternative agentic paradigms (pre-generated programs, policy orchestration) on the same tasks; no statistical tests; n=2 for throwing and n=5 for towel and experience conditions; transfer means rest on 1 vs 4 successful trials; and 20-50 minute executions at $7-31 per successful trial remain impractical.
Review complete
Assessment at a glance
5
risk flags identified
● high 2
● medium 2
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A general-purpose coding agent with frozen weights can serve as the full robot policy, achieving at least 80% success in seven of eight real-robot task configurations without task-specific or environment-specific training (Table 1).
- Keeping the agent in the runtime decision loop — choosing what to observe, writing and revising programs, and adapting to physical feedback — enables zero-shot manipulation beyond pre-generated programs and policy orchestration (Sections 1-3).
- Accumulating task experience as reusable files shortens execution time on repetition, with two-pair assembly task time falling 29.3% from the first to the fifth execution (Section 4.3.1, Table 7).
- Transferring frozen experience from GPT-6 Astra to the weaker GPT-5.6 Terra raises two-pair assembly success from 1/5 to 4/5 and cuts successful-trial mean time and tokens by 34.3% and 11.4% (Section 4.3.2, Figure 3).
Strengths
Genuinely novel policy boundary
Section 2.1 positions AGP as a third paradigm: unlike Code-as-Policies-style pre-generated programs (Liang et al. 2022) or VLA orchestration (Li et al. 2026a), the agent stays in the decision loop at runtime — deciding what to observe, authoring/revising programs, and adapting to physical feedback with frozen weights (Section 3.1, Eq. 1).
Real-robot breadth with pre-registered criteria
Five task groups span precision assembly of AutoMate parts with sub-millimeter clearances (Table 4: radial clearances 0.73-2.83 mm), dynamic timed-release throwing, and deformable bimanual folding; success criteria were fixed before evaluation (Table 3), and a candid failure analysis is included (Appendix H.1 tower collapse).
Transparent resource accounting
Fair-structure model/agent comparisons
Careful epistemic hedging
Weaknesses
No paradigm baselines
Statistical reporting absent
Subjective success criteria for deformables
Low reproducibility
Inclusion policy not itemized
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
AGP (Sections 1-3) positions a general-purpose coding agent as the entire robot policy: a preparation agent compiles a reusable task definition, then an execution agent with frozen weights interprets camera/depth evidence, writes Python programs for geometry and pose estimation, and issues motion/gripper commands through a calibrated seven-command bridge (move_ee, move_joints, gripper, frames, state, status, help; Appendix A.1). The claimed contribution is a third paradigm between program generation (Code-as-Policies) and policy orchestration: the agent stays in the decision loop at runtime. Experiments (Section 4) cover five real-robot task groups: video-guided four-pair assembly of AutoMate parts with sub-millimeter clearances (Table 4: radial clearances 0.73-2.83 mm), image-guided block construction, dice flipping, timed-release throwing, and bimanual towel folding. The headline claim — at least 80% success in seven of eight configurations (Table 1) — is well supported descriptively; claims of superiority over the alternative agentic paradigms are argued, not measured.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Agent as Policy (AGP)?
AGP is a framework in which a general-purpose coding agent (e.g., GPT-6 Astra running inside Codex) acts directly as a robot's policy: it reads camera and depth observations, writes Python programs to estimate object geometry, sends motion and gripper commands through a calibrated robot interface, and revises its actions based on physical feedback — with no task-specific training and fixed model weights.
How well does AGP perform on real robots?
On an I2RT YAM arm it succeeded in 8/10 four-pair assembly trials, 10/10, 10/10 and 9/10 on three block-construction configurations, 10/10 dice flips, 2/2 targeted throws, and 5/5 sequential plus 3/5 simultaneous towel folds — meaning at least 80% success in seven of the eight evaluated configurations.
What are the main drawbacks of AGP?
Execution is slow and expensive (roughly 20-50 minutes and 7-31 US dollars of inference cost per successful trial), deformable-object coordination is the weakest skill (3/5 on simultaneous folding), evidence for experience reuse and transfer rests on very small samples without statistical tests, and the paper provides no code release or baseline comparisons against alternative agentic paradigms.
Does accumulated experience make the agent faster?
In the paper's five-cycle study, reusing saved files of measurements, procedures, and scripts cut two-pair assembly execution time by 29.3% from the first to the fifth run, and giving the weaker GPT-5.6 Terra access to GPT-6 Astra's saved experience raised success from 1/5 to 4/5 — though the authors themselves note these comparisons need proper controls and rest on very few trials.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 5-12 and the qualitative success judgments they document could only be assessed from their captions; the underlying execution images/videos were not inspectable in the extracted text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.