GeoLAM: Learning Geometry-Grounded Latent Actions from Unlabeled Human Videos
Yifan Xie, Hekun Tian, Jinkun Liu, YuAn Wang, Qiao Sun, Wenbo Ding
cs.CV · cs.RO
Quick verdict
GeoLAM grounds latent actions in 4D geometry from unlabeled human video; LARYBench gains are large, but downstream margins are thin and variance unreported.
Summary
GeoLAM is a latent-action framework that pretrains on roughly 19,860 hours of action-free human (largely egocentric) video and transfers the representation to robot control. The encoder reconstructs future frames through a frozen geometric foundation model hierarchy (Depth Anything 3 features with a GLD reconstruction decoder), while a training-only 4D teacher (D4RT) supplies spatially pooled motion targets - 3D displacement, residual image-plane motion, and surface-orientation change - weighted by visibility, confidence, and observed motion magnitude. The resulting continuous latent action (four tokens, 32-dim deterministic bottleneck) serves as a denoising target in a Fast-WAM-style world-action model that jointly denoises latent actions and executable action chunks; future-video prediction is auxiliary, so neither the teacher nor video generation is needed at deployment. Reported results: best average regression MSE of 0.29 and 71.22% classification accuracy on LARYBench (vs 0.60 and 45.62% for LAPA-DINOv3), 98.5% average success on LIBERO, 92.6% on RoboTwin 2.0, and a 25-point OOD improvement over Fast-WAM in a 20-trial real-world bowl-placement task. Ablations isolate feature-level versus pixel reconstruction and the geometry loss. Main limitations: no variance or seed reporting for small benchmark margins, potentially confounded downstream baselines, an unexplained CALVIN anomaly in the ablation, and no code release.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A training-only 4D geometry teacher added to geometric-hierarchy future-frame reconstruction reduces LARYBench average regression MSE from 0.46 to 0.29 and raises average classification accuracy from 68.76% to 71.22% (Table IV).
- Latent actions learned exclusively from ~19,860 hours of action-free human video transfer to robot control, achieving 98.5% average LIBERO success and 92.6% average RoboTwin 2.0 success.
- The GeoLAM-based policy improves real-world OOD success by 25 percentage points over Fast-WAM (90% vs 65%) at comparable execution time (17.0 s vs 16.4 s).
- Deployment requires neither the geometry teacher nor future-video generation because both are used only during training.
Strengths
Clean ablation ladder isolating each component
Table IV separates feature-level reconstruction (F0 0.57, F1 0.54, F0+F1 0.49 avg MSE), pixel reconstruction through the frozen GFM hierarchy (0.46), and the added 4D-teacher loss (0.29), with matching accuracy gains 65.10% -> 66.58% -> 68.76% -> 71.22%, so the marginal value of Lgeo is quantified rather than asserted.
Deployment-efficient design consistent with its claims
The D4RT teacher and future-video generation are training-only; inference uses one clean-anchor Video DiT pass with cached key-values (Sec. III-B2), and real-world execution time is 17.0 s vs 16.4 s for Fast-WAM, empirically supporting the 'no test-time overhead' claim.
Large, unambiguous margins on the latent-action benchmark
Real-world OOD robustness evidence
Strictly action-free pretraining setting
Weaknesses
No statistical reporting for small downstream margins
Potentially confounded downstream comparison
Unexplained anomaly in the ablation
Reproducibility gaps despite no code release
Thin analysis of what the latent actions encode
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Does GeoLAM need the 4D geometry teacher at deployment?
No. The D4RT teacher is used only during human-video pretraining to shape the latent action space through a geometry-motion loss. Downstream training and inference use only the frozen latent-action encoder and the world-action model, so the teacher adds no test-time compute.
What data is GeoLAM pretrained on?
About 19,860 hours of action-free human video drawn from Egocentric-10K, EgoVerse, Ego4D, EgoLive, EgoDex, Something-Something V2, HoloAssist, and EPIC-KITCHENS-100, using RGB only - no action labels, hand poses, or camera parameters.
How much better is GeoLAM than previous latent-action methods?
On the LARYBench benchmark the paper reports an average control-regression MSE of 0.29 versus 0.60 for the strongest prior baseline (LAPA-DINOv3), a roughly 52% relative reduction, and 71.22% average action-classification accuracy versus 45.62%. On robot benchmarks it reports 98.5% average LIBERO success and 92.6% on RoboTwin 2.0.
Does GeoLAM generate future video when controlling the robot?
No. Future-video prediction is an auxiliary training task only. At inference the model runs a single anchor pass over the current observation and integrates a coupled flow to output an action chunk, keeping per-episode execution time (17.0 s in the real-world task) close to the Fast-WAM baseline.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-6 are only available as captions in the extracted text; qualitative reconstruction quality (Fig. 6) and corpus composition (Fig. 3) could not be visually verified.
- Several cited works (D4RT, LARYBench/LARY, Fast-WAM, LAWA, LAWM-3D) are 2026 preprints whose contents could not be checked against this review's text-only basis.
- No appendix was included in the extraction; some unspecified hyperparameters may be documented outside the provided text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.