TEMPO: Learning Temporal Context for Dynamic Robot Manipulation
Zhenyang Feng, Jimin Heo, Erik B. Sudderth, Unnat Jain, Unnat Jain
cs.RO · cs.CV · cs.LG
Quick verdict
Adds motion tokens + proprioceptive history to VLAs; strong real-robot gains across 4 dynamic tasks, though baselines are few and no error bars are reported.
Summary
The paper argues that vision-language-action (VLA) models fail at dynamic manipulation not because of latency or capacity but because single-frame observations lack temporal context. The authors identify two failure modes — motion ambiguity (a snapshot cannot predict where moving objects will be) and state aliasing (visually identical frames at different task phases require opposite actions) — and propose TEMPO, a plug-and-play augmentation of a pretrained VLA (built on the asynchronous VLASH backbone) that adds two inputs: TEMPOMOT, a motion token computed by cross-attending the current frame against a cached window of past-frame features from a frozen video encoder (default SAM 2.1-Tiny), and TEMPOACT, a mean-pooled 10-bucket summary of 300 recent proprioceptive commands. Only 2M parameters (0.08%) are added; inference rises from 33.8 to 35.3 ms. Across four real-robot tasks with 50 rollouts each, TEMPO reaches 74% on Bottle Handover (vs 44% for RTC), 66% on Drop Catch (below RTC's 80%), 66% on Flick Catch, and 98.1% on Wine Pour (mass retained); on untrimmed Flick Catch and Wine Pour it is the only method above 0% (68%/97.6%). Combined with UVT targets the average reaches 87.2%. Main limitations: only two baselines, no frame-stacking or flow-prediction comparison, no error bars or significance tests, and the streaming window length and demonstration counts are unspecified.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Single-frame VLA failures in dynamic manipulation decompose into two representational failures — motion ambiguity and state aliasing — that persist regardless of inference latency or model capacity (Sec. 1, 3).
- Two compact plug-and-play inputs (TEMPOMOT motion token + TEMPOACT proprioceptive history) resolve the respective failures without modifying the pretrained backbone, adding only 2M parameters and ~1.5 ms (Sec. 4, Tables 3, 5, 6).
- TEMPO improves Bottle Handover from 44% to 74% and is the only method above 0% on untrimmed Flick Catch (68%) and Wine Pour (97.6%), where both asynchronous baselines fail every rollout (Tables 1, 4).
- Trained-policy hidden states linearly decode object velocity far better with TEMPO (R²=0.57 on Flick Catch vs ≤0.03 for baselines), showing the motion signal is internalized (Table 7).
Strengths
Failure-mode-driven design with matched evidence
Section 3 formalizes motion ambiguity and state aliasing and maps each to a dedicated input; Table 6 (52%/56% single-signal vs 74% full), Table 1 (0% vs 68%/97.6% untrimmed) and Fig. A4 (TEMPOMOT critical at 2.7 s, TEMPOACT at 5.3 s) verify each signal addresses its own failure at a distinct episode phase.
Minimal, backbone-agnostic overhead
2M parameters (0.08%), +1.5 ms median latency (35.3 vs 33.8 ms, Table 3); Table 5 shows consistent gains across three backbones (π0.5 26%→56%, RTC 44%→80%, VLASH 38%→74% on Bottle Handover), including a non-asynchronous one.
Honest reporting of unfavorable and structural results
Mechanistic analysis beyond success rates
Released benchmark with per-frame motion labels
Weaknesses
No uncertainty quantification anywhere
Thin baseline set; the obvious temporal alternatives are missing
Reproducibility gaps in stated hyperparameters and assets
Table 4 'Average' conflates incommensurable metrics
'Regardless of model scale' claim rests on one model family
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
TEMPO (arXiv 2609.16864, UC Irvine) targets a well-motivated gap: pretrained vision-language-action (VLA) models condition on a single observation and therefore fail when objects move during manipulation. The conceptual contribution is a decomposition of this failure into two modes — motion ambiguity (a snapshot cannot predict scene evolution; Sec. 3, Fig. 2) and state aliasing (perceptually identical frames at different task phases demand opposite actions; Sec. 3, Table 1). The engineering contribution is TEMPO: two plug-and-play temporal inputs on a pretrained VLA — TEMPOMOT, a motion token from cross-attending current-frame features against a streaming cache of past-frame features from a frozen video encoder (SAM 2.1-Tiny default; Sec. 4.1), and TEMPOACT, a mean-pooled 10-bucket summary of the last 300 proprioceptive commands (Sec. 4.2). The claims are explicit and testable: the failures are representational, not latency or capacity failures; each signal fixes its own failure; and gains transfer across backbones.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is TEMPO in robot learning?
TEMPO is a plug-and-play augmentation for vision-language-action (VLA) robot policies that adds two temporal inputs: a motion token computed by cross-attending the current camera frame against cached past frames using a frozen video foundation model (SAM 2.1-Tiny), and a compact summary of the robot's last 300 proprioceptive commands pooled into 10 buckets. It targets dynamic manipulation — tasks where objects move during execution — and adds only 2M parameters (0.08%) and about 1.5 ms of inference time, without modifying the pretrained backbone.
What is 'state aliasing' in VLA models?
State aliasing is when visually near-identical observations occur at different points of a multi-stage task — for example, a gripper approaching an object versus releasing and retracting from it — but require opposite actions. A policy conditioned only on the current frame cannot tell the phases apart and oscillates or hesitates. In the paper, both asynchronous baselines score 0% on tasks containing such aliased phases, while adding a proprioceptive history input (TEMPOACT) resolves them.
What is TEMPO-Bench?
TEMPO-Bench is a benchmark released with the paper: over 50,000 annotated frames of human-robot dynamic manipulation with per-frame object-velocity labels, offered in two formats — regression (predict the object's velocity) and multiple-choice questions about motion direction and magnitude, following the MVBench and VLM4D formats — for evaluating motion understanding in vision-language and vision-language-action models.
How much does TEMPO improve robot task success?
On four real-robot dynamic tasks with 50 rollouts each, TEMPO reaches 74% on Bottle Handover (vs 44% for the RTC baseline and 38% for VLASH), 66% on Drop Catch (below RTC's 80%), 66% on Flick Catch, and 98.1% on Wine Pour (measured as bead mass retained). On the untrimmed versions of Flick Catch and Wine Pour, where both baselines score 0%, TEMPO achieves 68% and 97.6%. Combined with UVT training targets, the four-task average reaches 87.2%.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–5 and A1–A5 are available only through their captions in the extracted text; the visual evidence (e.g., Fig. 2 tracking plots) could not be independently verified.
- The TEMPOMOT streaming window length W never appears as a number in the extracted text; it may be omitted from the paper or buried in a figure, so this is treated as a reproducibility gap rather than an extraction artifact.
- arXiv v1 (15 Sep 2026) acknowledges CoRL feedback, so camera-ready changes may differ from this version.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.