CADSplat: Sparse-View 3D Gaussian Splatting Aided by CAD Models for Robust, Photorealistic Digital-Twin Reconstruction
Kristof Overdulve, Lode Jorissen, Nick Michiels
cs.CV · cs.GR
Quick verdict
Strong, honest sparse-view 3DGS method with CAD priors; CAD helps rendering mainly at 3 views, while constrained splats and poses drive most of the benefit.
Summary
CADSplat reconstructs photorealistic, geometrically consistent digital twins from sparse (3-15), wide-baseline posed images by regularizing 3D Gaussian Splatting with a retrieved CAD model. SAM3-segmented silhouettes are matched against offline-rendered silhouettes of a CAD library; a RANSAC Sim(3) consensus combines per-view hypotheses with the relative calibration to retrieve the model and register cameras to its canonical frame. 600k Gaussians are sampled on the mesh with frozen means/quaternions, warped by an 8x256 MLP deformation field, and trained in two stages: mask-aligned geometric warmup (pose, then pose+deformation) followed by photometric finetuning with SH-neighbor smoothness and splat-dropout regularizers. On SCO-CAD (6 objects) it leads all baselines (mPSNR 28.95 vs masked 3DGS 26.63, InstantSplat 27.90), degrades gracefully to 3 views, and on NeRS MVMC surpasses NeRS (18.7 vs 16.5 PSNR). Notably, a sphere initialization slightly beats CAD at full view budgets, with the CAD prior paying off only at 3 views and on self-occluded objects - an honest ablation that reframes the contribution toward the constrained representation plus pose and semantic benefits (2.9 deg median pose error, part transfer, physics simulation). Limitations: six-object primary dataset, dependence on relative calibration, non-metrology-grade deformed mesh, failure under extreme viewpoint extrapolation, no statistical testing, code promised but not yet released.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Anchoring a fixed set of ~600k splats to a retrieved, deformable CAD surface with silhouette-consensus registration achieves state-of-the-art rendering from 9-13 wide-baseline views and degrades gracefully to 3 views (IoU 0.942 to 0.929; mLPIPS 0.031 to 0.083).
- Most of the rendering gain over unconstrained 3DGS comes from the constrained formulation - surface-anchored splats moved by one smooth shared deformation field - not from the CAD shape at full view budgets (sphere init: 23.54 vs 23.24 PSNR).
- The CAD prior contributes where views are scarcest and occlusion strongest: at 3 views it yields +1.4 dB PSNR and +4.9 IoU points over sphere initialization.
- The pipeline recovers accurate camera-to-object poses (median 2.9 deg rotation, 6.1% center error after refinement), enabling markerless AR registration, per-image pose estimation, CAD part-label transfer, and physics simulation.
Strengths
Honest reporting of an adverse ablation
Table 5 openly shows sphere initialization beating CAD initialization on every metric at the full 9-13-view budget (PSNR 23.54 vs 23.24, mPSNR 29.40 vs 28.95, IoU 0.946 vs 0.942), and the authors precisely localize where the CAD prior does pay off: at 3 views (+1.4 dB PSNR, IoU 0.929 vs 0.879) and on voluminous self-occluded objects. This candor strengthens credibility.
Careful, self-aware evaluation methodology
Section 4.3 explicitly diagnoses the calibration-mismatch problem (fig. 6: few-pixel silhouette bands dominating pixel metrics) and responds with eroded-mask metrics, fgPSNR, silhouette IoU, and dual reporting with and without 400-step test-time pose refinement (tables 2 and 3), so readers can separate pose alignment from reconstruction quality.
Genuinely sparse capture protocol
Complete component-wise ablation
Value beyond rendering metrics is demonstrated, not just claimed
Weaknesses
Headline margins depend on test-time pose refinement
Small primary benchmark with no variance treatment
CAD prior's rendering contribution is negligible at the headline operating point
No retrieval accuracy or failure analysis
Registration ground truth is partly lenient and manual
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Does the CAD model actually improve reconstruction quality in CADSplat?
Only in the sparsest settings. At the paper's main 9-13-view setting, replacing the CAD mesh with a plain sphere initialization scored slightly better on every reported metric (e.g., PSNR 23.54 vs 23.24, mPSNR 29.40 vs 28.95). The CAD prior pays off at 3 views (+1.4 dB PSNR; silhouette IoU 0.929 vs 0.879) and on strongly self-occluded objects. Its more consistent value is providing camera-to-object poses, CAD part semantics, and a collision-ready mesh, which a sphere cannot offer.
What datasets was CADSplat evaluated on?
Three: SCO-CAD, a new dataset of 6 real-world sparse captures of common objects (skateboard, scissors, hammer, carjack, senseo machine, a car) paired with approximate public CAD models, to be released with code; NeRS MVMC, 20 casually photographed cars with a 3,533-model ShapeNetCore car library; and NeRS Misc, 8 household objects evaluated qualitatively only.
How many images does CADSplat need to reconstruct an object?
It targets sparse, wide-baseline captures: 9-13 images give the best results and quality degrades gracefully down to 3 images - silhouette IoU falls only from 0.942 to 0.929 while textures blur (mLPIPS grows 2.7x). One caveat from the paper: the 3/5/8-view experiments reuse the calibration of the full capture, so they model a pre-calibrated multi-camera rig rather than calibrating from 3 photos alone.
Is code or data available for CADSplat?
The paper states that the six SCO-CAD objects, their CAD models, and the full pipeline code 'will be released'; at the time of this review no public repository had been verified. The method builds on the open-source gsplat framework and uses the public SAM3, HLoc, FSGS, InstantSplat, and NeRS components.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Qualitative comparisons (figs. 7, 10, 11) and the failure case (fig. 14) cannot be assessed from extracted text; visual-quality judgments rely on captions and metric tables.
- Online Resources 1-4 (novel-view and application videos) were not reviewable.
- Full per-scene result tables are not present in the extraction; per-scene statements (e.g., gaps concentrated on car1/carjack/senseo) are taken from prose.
- Baseline rows for NeRS and CADSim are transcribed from prior publications per the authors; protocol equivalence is assumed as claimed and could not be independently verified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.