StrucPhysVideo: Learning Physical Dynamics from Structured Captions and Robot Actions
Awomo-WM Team, :, Enhui Ma, Kaiwen Guo, Tingrui Zhang, Wei Song, Yingshui Tan, Jianhua Xu, et al.
cs.CV
Quick verdict
Claims 45.5% Physics-IQ SOTA via structured physics captions and action-conditioned video models; but single-run evals and a private dataset need scrutiny.
Summary
This paper introduces StrucPhysVideo, a family of video world models linking physics-focused data curation with language- and action-conditioned generation. The pipeline splits source videos at shot boundaries, selects clips using motion-energy-based window placement, filters technical defects and impure content (faces, screens, overlays vs. scene text), then applies two-call VLM annotation (Qwen3.6-27B): physical verification plus structured captions separating camera motion, world knowledge, and timestamped entity behaviors, followed by closed-set tagging across a 26-phenomenon physics taxonomy. StrucPhysVideo-TI2V is a ~30B sparse-MoE DiT fine-tuned from LingBot-Video with frozen Qwen3-VL-32B conditioning and first-frame-constrained flow matching, trained on 20,000 hours under a physics-weighted curriculum; it reports 45.5% on Physics-IQ Verified, 2.8 points above Cosmos3-Super. Caption ablations on two backbones show structured captions beating raw captions (+1.6/+2.2 points). StrucPhysVideo-IA2V adds a causal action encoder and three-phase causalization/DMD distillation for four-step action-conditioned rollouts, outperforming DreamDojo-AgiBot and A2World on AgiBot metrics. Main limitations: single-run comparisons without variance estimates, mixed provenance of reproduced versus leaderboard scores, a private 20k-hour corpus, and confounded IA2V baselines differing in action space and training data.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- StrucPhysVideo-TI2V scores 45.5% on Physics-IQ Verified, the highest among 17 compared models on the 16 September 2026 snapshot, exceeding Cosmos3-Super Image2Video by 2.8 percentage points.
- Physics-focused structured captions outperform raw captions on Physics-IQ Verified across two backbones (Wan2.2-5B: 31.0 to 33.2; LingBot-Video-30B: 43.9 to 45.5), and both beat reproduced baselines.
- StrucPhysVideo-IA2V outperforms DreamDojo-AgiBot on all reported metrics and A2World under matched 256x256 settings (e.g., trajectory accuracy 0.8330 vs 0.7905).
- The curation pipeline's structured, temporally localized annotations, which separate camera motion from object behavior, provide supervision grounded in observable physical events.
Strengths
End-to-end curation pipeline with unusually explicit design rationale
Sections 3.2-3.3 name concrete components (PySceneDetect+TransNetV2 fusion, flow-energy clip placement in Eq. 22, YuNet faces, CLIP screen screening, EasyOCR text tracks, Qwen3-VL-4B five-class text review) and motivate each rejection class, including the scene-text-versus-overlay distinction that naive text filters get wrong (Fig. 8).
Clean attribution design for the caption ablation
Figure 18 holds the dataset fixed and swaps only caption type across two backbones (Wan2.2-5B 31.0 to 33.2; LingBot-Video-30B 43.9 to 45.5), with reproduced baselines marked with *, isolating the supervision variable rather than reporting only a whole-system comparison.
Complete, reproducibility-oriented method description
Candid scoping and annotation hygiene
Informative negative results in the guidance ablation
Weaknesses
No statistical uncertainty for any comparison
Reproduced-versus-leaderboard inconsistency weakens the headline margin
Dataset scale and composition unverifiable
Confounded IA2V baseline comparison
Caption ablation scoped to one benchmark with a partially confounded contrast
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What score does StrucPhysVideo achieve on Physics-IQ Verified and how does it rank?
The paper reports 45.5%, the highest among the 17 models in its comparison based on a benchmark snapshot dated 16 September 2026. It is 2.8 percentage points above Cosmos3-Super Image2Video (42.7%) and 9.3 points above the strongest closed-source comparator listed, MiniMax H3 Max (36.2%).
What data was StrucPhysVideo trained on?
About 10,000 hours each of physics-focused and general-domain internet video (20,000 hours total), curated via shot-boundary splitting, motion-aware clip selection, technical quality and content filtering, then annotated with structured captions and a closed-set physics taxonomy of 26 leaf phenomena. The corpus is described as privately collected; the action-conditioned model additionally trains on AgiBot World-Beta robot data plus ~500 hours of general video.
Is the code, data, or model for StrucPhysVideo publicly available?
The paper lists a project page (westlakedi-awomo.github.io/StrucPhysVideo-Page) and a GitHub repository (github.com/westlakedi-awomo/StrucPhysVideo), but the text does not state which weights, datasets, captions, or pipeline components are actually released, and the 20,000-hour training corpus is described as privately collected.
What is the difference between StrucPhysVideo-TI2V and StrucPhysVideo-IA2V?
TI2V generates video from a reference image plus a text caption using a ~30B sparse mixture-of-experts diffusion transformer. IA2V instead predicts how a scene evolves from a reference image plus robot end-effector action trajectories, using a causal action encoder, autoregressive chunked rollout, and four-step distilled sampling for interactive generation; a TIA2V variant also accepts a text command.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure contents (Figs. 1-19) could not be analyzed as images; leaderboard values for models other than those named in the prose were inferred from the ordered OCR text of Figure 1.
- Figure 12's filtering survival counts are referenced but their numeric values are absent from the extracted text, limiting assessment of dataset scale and composition.
- The abstract in the metadata was truncated mid-sentence; claims were reconstructed from the complete Section 1 and abstract-like prose in the body.
- The GitHub repository and project page were not inspected (offline review), so the actual code/data release status behind the stated links is unverified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.