OmniFysics-Nano-V2 Technical Report: Understanding the Physical World Across Modalities
Yizhou Liu, Jinghang Han, Kaixiang Qiu, Qi He, Minghao Han, Yue Jiang, Xujia Chen, Wei Zou, et al.
cs.AI
Quick verdict
4B omni model with physics-aware data and 2-stage GRPO claims 17/21 benchmark wins; strong but top omni rivals untested and 2 benchmarks self-published.
Summary
OmniFysics-Nano-V2 is a 4B-parameter omni-modal model — initialized from Qwen3.5-4B, a Whisper Medium audio encoder, and CosyVoice3 speech decoders — that accepts image, video, audio, speech, and text input and generates text and speech. Targeting two claimed gaps in omni-modal training (absent physical supervision, homogeneous objectives), the authors build a dual-branch data pipeline: a static branch grounds salient objects in structured physical attributes estimated by a commercial MLLM and checked against a 1,209-entry object-material prototype bank with range filtering and a single re-estimation pass, while a dynamic branch aligns event-centered video clips with acoustic events and interaction outcomes. RL prompts are curated by reward-diversity filtering (four rollouts per prompt; keep only prompts with more than one distinct reward), reported to remove 44–74% of prompts per category and save roughly 2,688 GPU-hours, followed by a two-stage GRPO curriculum: answer correctness first, then fine-grained rewards on intermediate physical perception and format. The model reports best-in-table results on 17 of 21 benchmarks (e.g., FysicsEval Understanding 98.27, PhysUniBench 59.42, Daily-Omni 85.24). Key limitations: two benchmarks come from the same lab, the strongest recent omni baselines (Qwen3-Omni, Qwen3.5-Omni) are absent, physics labels are teacher-generated without accuracy audits, there is no variance or significance reporting, and the embodied physics-engine section lacks quantitative results.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 3
● low 1
6 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A compact 4B omni-modal model trained with a dual-branch physics-aware data pipeline and two-stage GRPO achieves best-in-table results on 17 of 21 benchmarks, e.g., FysicsEval Understanding 98.27 (+5.57 over the best baseline) and PhysUniBench 59.42.
- Static property supervision mainly improves property-oriented benchmarks while dynamic event supervision drives temporal and cross-modal gains (Table 5).
- Reward-diversity filtering (K=4 rollouts; keep only prompts with >1 distinct reward) removes 44-74% of RL prompts per category and saves an estimated 2,687.7 GPU-hours without losing learning signal.
- Two-stage RL is necessary: answer-level GRPO alone degrades OmniBench and PhysBench, and fine-grained intermediate-perception rewards recover and exceed the SFT checkpoint (Figure 10).
- Answer-conditioned speech generation is preferred over native CosyVoice3 in pairwise human evaluation by 32.8/17.4/42.4 percentage points on naturalness/clarity/semantic faithfulness.
Strengths
Formalized, guarded data pipeline
§3.1 specifies the static branch end-to-end (Eqs. 1-3): commercial-MLLM property estimates retrieved against a 1,209-entry handbook prototype bank via Jaccard keyword matching, range-checked, re-estimated once, and discarded if still invalid; the dynamic branch (§3.1.2) aligns visual and acoustic onsets within an event-dependent tolerance δ (Eq. 5). The guardrails make label provenance explicit rather than hand-waved.
Reward-diversity filtering is simple and reusable
§3.2 (Eqs. 6-7) drops prompts whose K=4 rollouts yield a single reward value — exactly the degenerate groups that give zero GRPO advantage (Eq. 17). Table 7 quantifies removal per category (73.99% math, 69.79% video, 57.69% multimodal, 44.15% image-text), making the curation auditable.
Ablations are unusually candid
Intermediate-perception rewards address a real RL weakness
Size-honest comparison set
Clean speech-conditioning design
Weaknesses
Strongest omni baselines missing
Two of 21 benchmarks are self-published
No uncertainty quantification
Teacher-generated physics ground truth without audit
Physics-engine sections lack any quantitative result
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper presents OmniFysics-Nano-V2, a 4B omni-modal model (image/video/audio/speech/text input; text and speech output), assembled from Qwen3.5-4B, a Whisper Medium audio encoder, and CosyVoice3 speech decoders (§5.1). It argues omni-modal training suffers two deficits — absent physical supervision and homogeneous objectives (§1) — and responds with: (1) a dual-branch data pipeline (static object-property grounding; dynamic audio-visually aligned events, §3.1); (2) reward-diversity filtering of RL prompts (§3.2); and (3) a two-stage GRPO curriculum (§4.2.2). The central empirical claim is best-in-table performance on 17 of 21 benchmarks against the listed baselines, headlined by FysicsEval Understanding 98.27 (+5.57 over Qwen3.5-4B) and PhysUniBench 59.42 (Table 3).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 6 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is OmniFysics-Nano-V2?
It is a 4-billion-parameter open omni-modal AI model from the Physical Superintelligence Lab (Fysics AI / Fudan University) that processes images, video, audio, speech, and text in one model and can respond in either text or synthesized speech. It is trained with extra 'physical-world' supervision — structured object attributes like mass, density, and stiffness, plus audio-visually aligned event data — and a two-stage reinforcement-learning curriculum (GRPO). The paper reports best-in-table scores on 17 of 21 public benchmarks.
Does it really beat larger models like Qwen2.5-Omni-7B?
On the 21 benchmarks listed in the paper it beats Qwen2.5-Omni-7B on most, with the largest gains on Daily-Omni (+31.82), Video-MME (+16.59), and WorldSense (+14.92). However, it loses on some general benchmarks (MMMU 62.11 vs 77.60 for the 4B Qwen3.5 VL model; MMBench-V1.1 and AI2D trail OmniVinci; AIME26 trails Qwen2.5-Omni-7B), and newer Qwen3-Omni / Qwen3.5-Omni models are not compared at all, so 'beats SOTA omni models' should be read cautiously.
Is the model or code available?
The paper lists a GitHub repository (github.com/Fysics-AI/OmniFysics-Nano-V2) and Hugging Face weights (huggingface.co/Fysics-AI/OmniFysics-Nano-V2). However, the physics data pipeline depends on an unnamed commercial MLLM, and the embodied section uses an in-house differentiable physics engine that does not appear to be released, so full training reproduction from the paper alone is not possible. Independent evaluation of released weights on public benchmarks is the practical verification route.
How does the model estimate physical properties like mass from an image?
During training-data construction, a commercial multimodal LLM estimates each object's category, material, and numeric properties (mass, density, stiffness, Young's modulus, etc.); estimates are checked against a prototype bank of 1,209 object-material entries compiled from five material handbooks, and out-of-range values get one re-estimation pass before the sample is kept or discarded. The released model learned to emit such structured estimates from this data. The paper itself notes these outputs are prior-informed estimates, not measurements, and that quantitative reliability is judged by the physics benchmarks.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure text is partially garbled by extraction (Figs. 2, 3, 6, 7, 11); chart values in Figs. 8-10 were readable from embedded labels but layout-dependent readings may be imperfect.
- External artifacts (GitHub/HF links, leaderboard entries) could not be verified offline; 'codeAvailable' reflects the paper's stated links only.
- An appendix with hyperparameters may exist outside the extracted text; several training details (λ weights, group size) were not found in it.
- The scoring scale for FysicsEval Prediction/Reasoning (all models ~2-3) is ambiguous from the extracted text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.