Show-Harness: Just a VLM Agent Can Play Robots
Yanzhe Chen, Zechen Bai, Zhijun Cao, Wenzheng Zeng, Kevin Qinghong Lin, Yiqi Lin, Guoqiang Liang, Kevin Yuchen Ma, et al.
cs.RO · cs.AI · cs.CV · cs.MM
Quick verdict
Claims semantic action units let VLMs control robots zero-shot (89% vs 57% best baseline); strong but small-N, no significance tests, latency unreported.
Summary
Show-Harness is an embodied harness that lets vision-language models (VLMs) control robots through a compact discrete semantic action interface — six translation units, two rotation units, GRASP/RELEASE/DONE — where embodiment-specific interpreters deterministically ground each unit into low-level control. The paper evaluates two modes on real Franka and AgileX arms across ten object–receptacle pick-and-place tasks: zero-shot use of Gemini-3.1 Pro (89% average success versus 57% for the strongest baseline RATS and 39% for π0.5), and LoRA fine-tuning of a Qwen3.5-2B on 164 keyboard/VLM-collected demonstrations (86%, under 2 GPU-hours), which also achieves sim-to-real transfer (13/20) where fine-tuned VLAs score 0/20. GUMI, a GUI over the same action space, enables teleoperation-free demonstration collection by humans and agents. Targeted scenarios show interpreter-only adaptation to finer step sizes, new action compositions, and unseen rotation angles, plus plugin ablations attributing gains to multi-view guidance, proprioception, and planning. An action-representation ablation finds written conventions, not action names, provide most of the grounding. Main limitations: only 10 trials per task with no statistical tests or confidence intervals; wall-clock latency and API cost of per-step frontier-model calls are largely unreported against fast VLAs; the task suite is narrow pick-and-place; and heavily engineered prompt conventions mean much of the 'zero-shot' capability rests on hand-written interface design rather than VLM intelligence alone.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 4
● low 0
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A compact discrete semantic action interface with deterministic per-embodiment interpreters lets frontier VLMs control real robots zero-shot.
- Zero-shot Gemini-3.1 Pro averages 89% across ten real manipulation tasks, vs 57% for the best of six baselines (RATS) and 39% for fine-tuned π0.5 (Table 2).
- A Qwen3.5-2B with LoRA (<2 GPU-hours, 164 GUMI demonstrations) reaches 86% and achieves sim-to-real transfer (13/20) where fine-tuned VLAs score 0/20.
- Physical adaptability (finer steps, new action compositions, unseen rotation angles) is obtained by editing the interpreter alone, without retraining (§5.3.1).
- GUMI enables demonstration collection in the same action space by humans and agents without teleoperation hardware, reusable across embodiments.
Strengths
Matched-data controlled comparison
Trainable baselines (π0.5, GR00T) are fine-tuned on the same 164 GUMI demonstrations converted to continuous trajectories (§5.1), making the VLA-versus-interface comparison controlled rather than anecdotal.
Unusually thorough ablation program
Nine-plugin leave-one-out (Fig. 9), backbone scaling 0.8B-9B (Fig. 8), a 2×2 action-representation design (Fig. 10), and backbone × thinking-effort analysis (Fig. 7) systematically isolate design choices.
Self-limiting attribution finding
Full prompt and data release
Sim-to-real under matched conditions
Weaknesses
Sample sizes cannot support per-task claims
Wall-clock and cost absent from main results
Narrow task distribution
VLA baselines evaluated in a low-data regime
Heavy prompt engineering behind 'zero-shot'
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper introduces Show-Harness, a model-agnostic embodied harness that lets vision-language models control robot arms through a compact discrete semantic action interface: six translation units (MV_FWD/BACK/LEFT/RIGHT/UP/DOWN), axis-paired rotation units, and GRASP/RELEASE/DONE. Embodiment-specific interpreters deterministically convert each unit into Cartesian setpoint updates (Eq. 4) with workspace limits, so the VLM owns decisions while the interpreter owns metric execution. Two modes are claimed: zero-shot frontier control (Gemini-3.1 Pro) and few-GPU-hour LoRA adaptation of a 2B open VLM (Qwen3.5-2B). A companion tool, GUMI, exposes the same action space as a GUI for teleoperation-free demonstration collection. Headline claims: 89% average zero-shot success versus 57% for the strongest of six baselines (Table 2), 86% fine-tuned, sim-to-real 13/20 versus 0/20 for fine-tuned VLAs, and interpreter-only adaptation to precision, composition, and rotation changes.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Show-Harness?
It is a framework that lets vision-language models operate robot arms through a small set of named, discrete actions (move in six directions, rotate, grasp, release, done). Each robot has an interpreter that deterministically converts those symbolic actions into precise low-level motions, so the same model interface works across different robots.
Does Show-Harness require training?
Not for the frontier-model mode: Gemini-3.1 Pro controls the robot zero-shot (89% average success over ten real pick-and-place tasks). A small open model (Qwen3.5-2B) can be LoRA-fine-tuned in under 2 GPU-hours on 164 demonstrations to reach 86%.
How does it compare to VLA models like π0.5?
In the paper's matched-data comparison on real arms, Show-Harness scores 89% (zero-shot) and 86% (fine-tuned) versus 39% for fine-tuned π0.5 across ten tasks. Caveats: only 10 trials per task and no statistical significance tests are reported.
Is code or data available?
The paper links a project page with code, model, and dataset, and releases the demonstration corpus on Hugging Face (showlab/Show-Harness-Data); full prompts are published in the appendix.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure-embedded values (Figs. 6-10) were extracted as unlabelled digit strings; this review prioritizes Table 2 and in-text prose numbers and avoids quoting ambiguous figure readings.
- The extraction ends at the fine-tuned VLM prompt block, which reads as the natural end of the paper; no missing continuation is claimed.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.