DiagGen: Agentic Generation of Deformable Assets with Sim-based Diagnostics for Robotic Simulation
Guanxiong Chen, Yiduo Qu, Qianjun Xia, Pengyu Jing, Yixian Cheng, Bole Ma, Pengzhi Yang, Bingyang Zhou, et al.
cs.RO · cs.AI
Quick verdict
DiagGen closes the loop from image to simulatable deformable via sim-based stage-routed repair; credible but small-n, VLM-judged, no significance tests.
Summary
DiagGen is an agentic framework that turns a single in-the-wild photograph into a simulation-ready deformable asset via a generate–simulate–diagnose–refine loop. After image cleanup, SAM3/OmniPart segmentation, part-wise material inference, and mesh processing, a VLM diagnostic agent probes semantically chosen anchor regions in the Genesis FEM simulator with grasp–displace–release episodes and routes implausible responses to the segmentation, material-inference, or mesh-processing stage for targeted repair. Of 40 real-image assets, 14 were flagged for revision: PMSC rose from a 2.14 to a 3.43 mean (10 improved, 4 tied, none worsened), and paired PAIP comparisons favored the revised asset in 9 of 14 cases (4 baseline wins, 1 tie). Three seeded-fault case studies (curling iron, massager, kitchen tongs) all yielded correct issue identification and stage routing. Demonstrations include pick-and-place trajectory optimization in Genesis, monolithization gains over raw OmniPart meshes, and integration into an Agentic Real2Sim digital twin. Limitations: evaluation rests entirely on VLM judges (GPT-5.6 Terra judging GPT-5.6 Luna outputs) with no human ratings or significance tests; asset-selection and retrospective PMSC protocols are under-specified; and large-scale generation remains untested.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 4
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Embedding a physics simulator inside the generation loop lets a VLM diagnostic agent attribute implausible behavior to specific upstream stages and route targeted repairs (C1, Sec. III-B–C).
- On 40 real-image assets, diagnostics-guided revision moderately improves quality: PMSC rose from 2.14 to 3.43 on the 14 revised assets, and PAIP favored revisions in 9 of 14 paired comparisons (Sec. IV-B).
- Seeded faults in material inference, segmentation, and mesh processing are correctly identified and routed in all three case studies, with no fault information leaked to the agent (Sec. IV-A, Table I).
- Unlike raw visual-foundation-model outputs, DiagGen assets drop directly into high-fidelity Genesis simulation for contact-rich pick-and-place and scene-level Real2Sim digital twins (C3, Figs. 4–6).
Strengths
Simulator promoted from validator to diagnostic instrument
Unlike PhysX-Omni's post-generation tests (Sec. I, II), the diagnostic agent plans asset-specific probes (semantic anchors, grasp boxes, approach–contact–displace–release protocol, Sec. III-B) and uses observed simulator responses plus artifact records to attribute faults to owning stages — a genuinely new control loop for asset generation.
Seeded-fault validation with ground truth and leakage control
Sec. IV-A / Table I: three injected faults (cord given barrel stiffness, truncated coil mask, metric scale 0.28→2.8 m) map one-to-one to the three repair routes; the agent received no fault information and correctly identified both issue and route in 3/3 cases.
Candid, well-hedged reporting of mixed outcomes
Above-average evaluation hygiene within the VLM-judge paradigm
Demonstrations target the actual failure mode of raw generative meshes
Weaknesses
Quantitative claims rest entirely on VLM judges
No statistical inference and small samples
Missing matched-compute baseline
Retroactive, under-specified PMSC protocol
Reproducibility depends on proprietary frontier models and an uncurated image set
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
DiagGen (arXiv 2609.23103, cs.RO) attacks the gap between visually plausible generated 3D assets and assets that behave correctly in a physics engine. The authors frame two deficits in prior work: agentic pipelines validate only after generation (PhysX-Omni), and deformable-reconstruction approaches (PhysTwin, EMPM, DSO, PokeFlex) need calibrated interactions or high-fidelity scans (Sec. I–II). The claimed contributions are (C1) an agentic generate–simulate–diagnose–refine workflow producing deformables from a single image, (C2) a diagnostic agent that attributes failures to upstream stages and routes repairs, and (C3) demonstrations that the assets support contact-rich manipulation and scene-level Real2Sim. Headline numbers: over 40 real-image assets, 14 were routed to revision; PMSC rose from a 2.14 to a 3.43 mean (gain 1.29; 10 up, 4 ties, 0 down); PAIP paired comparisons give 9 revised-wins, 4 baseline-wins, 1 tie; three seeded faults were all correctly diagnosed and routed (Sec. IV, Tables I–II).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does DiagGen do?
DiagGen turns a single ordinary photo of a deformable object (a plush toy, kitchen tongs, a cable) into a 3D asset that behaves plausibly in a physics simulator. An AI-agent pipeline segments the object into parts, assigns each part material properties, and then a diagnostic agent physically probes the object in the Genesis simulator by grasping, displacing, and releasing it; if a behavior is implausible, the pipeline routes a repair to the responsible earlier stage (segmentation, material inference, or mesh processing) and regenerates.
How well does DiagGen perform?
On 40 real-image assets, the diagnostic agent recommended revisions for 14. Retrospective PMSC scores rose from a 2.14 to a 3.43 mean on a 5-point scale (10 improved, 4 tied, none worsened), and in paired simulator comparisons (PAIP) the revised asset won 9 of 14 times (4 baseline wins, 1 tie). Three deliberately broken assets (curling iron, massager, kitchen tongs) were all correctly diagnosed and routed to the right repair stage. Note the evaluation is done by VLM judges, not humans, and no significance tests are reported.
Which models and simulator does DiagGen use?
GPT-5.6 Luna for generation and diagnostics, GPT-5.5 (high reasoning) for seeded-fault diagnostics, GPT-5.6 Terra as the evaluation judges, and GPT-5.6 Sol for trajectory optimization. SAM3 and OmniPart handle segmentation and part geometry, and the Genesis FEM simulator with SAP frictional contact is used for probing and manipulation tests.
Is DiagGen's code or data available?
The paper points readers to accompanying source code and a project website (diaggen.github.io). However, the pipeline relies on proprietary GPT-5.6-family models, and the 40 test images come partly from the public MVImgNet2.0 dataset and partly from unspecified personal phone photos, so full replication requires the authors' release plus paid API access.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- All figures (Figs. 1–6) are unrecoverable from extracted text, so visual plausibility claims could not be inspected directly.
- Tables I and II were flattened into prose in the extraction; agent quotes were checked against that prose, but original table formatting was unavailable.
- Full prompts, skills, and diagnostic schemas live in the 'accompanying source code' and could not be verified from the paper text.
- Code-release terms and the 40-image asset list are not stated in the text; availability judgments rest on the paper's own references to accompanying code and the project website.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.