PhysVGGT: Feed-Forward Dense Physical Property Estimation from A Single Image
Sneha Paul, Guile Wu, Bingbing Liu, Dongfeng Bai
cs.CV
Quick verdict
Single-image feed-forward physics + mass prediction (0.13s, claimed SOTA), but supervision is a VLM material lookup and ablation gains sit within seed variance.
Summary
PhysVGGT is a feed-forward transformer that predicts dense per-pixel maps of friction, Shore hardness, Young's modulus and density, plus object-level mass, from a single RGB image in a 0.13 s forward pass. A frozen 1.26B-parameter StreamVGGT backbone supplies geometry-aware tokens; a trainable 32.4M head combines a DPT feature extractor, a shared UNet-style encoder-decoder with a cross-property coupling attention module at the bottleneck, lightweight per-property heads, and a pooled-token MLP that regresses log-mass. Supervision comes from a pseudo-label pipeline over ABO-8k (8,213 objects, ~123k renders): SAM 2 part segmentation, VLM material/Shore-hardness assignment, cross-view majority-vote refinement, then material-property table lookup rasterized per pixel. Reported results: mass ADE 4.73 kg on ABO-500 (best among quoted baselines), zero-shot best on all friction and hardness metrics on the OOD NeRF2Physics real scenes, competitive zero-shot mass on image2mass, and 27x speedup over VoMP. Key limitations: supervision is essentially a 12-material lookup table, so dense outputs are piecewise-constant per part; no statistical significance testing and seed variance exceeds some ablation deltas; the pseudo-label VLM is unnamed and no code release is stated; dense-property accuracy is evaluated only qualitatively.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- PhysVGGT predicts dense friction, Shore hardness, Young's modulus, and density maps plus object-level mass from one RGB image in a single 0.13 s forward pass, 27x faster than the fastest prior method (VoMP).
- A SAM 2 + VLM pseudo-label pipeline with cross-view refinement enables large-scale weakly supervised dense-property training without physical measurements, and refinement improves every property (Table 14).
- On ABO-500 mass estimation, PhysVGGT achieves ADE 4.73 kg, improving on the best prior result (GaussianProperty, 5.96 kg) while using a single view at inference.
- Zero-shot on the out-of-distribution NeRF2Physics real benchmark, PhysVGGT is best on all friction and hardness metrics (e.g., friction ADE 0.102 vs 0.209 for GPT-4V) and remains competitive on image2mass without fine-tuning.
Strengths
Large efficiency gain over the prior paradigm
Single 0.13 s forward pass with no test-time VLM or per-object reconstruction (Table 3), 27x faster than VoMP (3.6 s) and ~4 orders faster than NeRF2Physics (1454.6 s); only 32.4M trainable parameters on a frozen 1.26B backbone (Table 6).
Scalable supervision pipeline with measurable refinement benefit
The SAM 2 + VLM + cross-view vote pipeline (Sec 3.3) yields ~123k densely labeled renders from ABO-8k without 3D assets or physical measurements; Table 14 shows refinement improves friction MnRE 0.813→0.831, hardness 0.856→0.908, dense mass 0.450→0.655.
Unusually thorough ablation coverage
Evaluation extends beyond in-distribution renders
Disciplined benchmark hygiene in one instance
Weaknesses
Supervision collapses physical properties to material classification
No statistical rigor; ablation deltas may be seed noise
Reproducibility gaps in the data engine
Baseline comparisons are partly quoted, not re-run
Core dense-property claim rests on qualitative evidence
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does PhysVGGT do?
PhysVGGT is a neural network that takes a single RGB photo of an object and, in one forward pass (~0.13 seconds), outputs dense per-pixel maps of four physical properties — friction coefficient, Shore hardness, Young's modulus (stiffness), and density — plus an estimate of the object's total mass. It needs no 3D reconstruction or test-time language-model queries, unlike earlier methods that rebuild a 3D model of each object first.
How much faster is PhysVGGT than previous physical-property estimation methods?
The paper reports 0.13 s per image. For comparison, VoMP (the previous fastest feed-forward method) took 3.6 s (27x slower), AURA 4.7 s, and NeRF-based pipelines like NeRF2Physics about 1,455 s per object — roughly 11,000x slower. PhysVGGT achieves this by training only ~32M parameters on top of a frozen 1.26B-parameter geometry backbone (StreamVGGT).
Where does PhysVGGT's training data come from — are the labels real physical measurements?
No. The labels are pseudo-labels generated automatically: SAM 2 segments object parts in multi-view product renders from the Amazon Berkeley Objects catalogue (~8,200 objects, ~123k images), a vision-language model guesses each part's material and Shore hardness, cross-view voting fixes inconsistencies, and numeric property values are looked up from standard engineering material tables. The only real measurements used are catalogue item weights for mass and the small NeRF2Physics benchmark used only for zero-shot testing.
Does PhysVGGT work on real-world images, not just product renders?
It was trained only on synthetic product renders, but the paper reports zero-shot tests on real data: on the NeRF2Physics real-scene benchmark it beats all reproduced baselines (including GPT-4V and CLIP) on friction and hardness metrics, and on image2mass real photos it reaches a mass-ranking score (MnRE 0.653) competitive with models trained in-domain, though it trails shape-aware baselines and median human estimates. Terrain generalization (RUGD, RELLIS-3D) is shown only qualitatively, without measured ground truth.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-8 could not be visually assessed from the extracted text; claims about dense-map quality (Figs 4, 5) and the grasp-force demo (Fig 8) rest entirely on figures not verifiable here.
- Table 4 contains a stray '−' column with no header, likely a text-extraction artifact; the qualitative analysis of that table assumed only the reported MnRE values.
- A few prose-vs-table ambiguities were resolved by inference from Table 1 (e.g., the '43%' improvement in Sec 4.1 refers to ADE vs AURA, not ALDE).
- Minor internal inconsistencies exist (trainable parameters stated as 32M/31.9M/32.4M in different places; Fig 7 VLM-generated text contains typos); these do not change the assessment but signal incomplete copy-editing.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.