PosteriorBench: From Point Estimates to Posterior Matching in Evaluating Generative Inverse Solvers
Jiachen Yao, Zi-Siang Hsu, Xi Deng, Aditi Gupta, Xin Ju, Sally M Benson, Gege Wen, Anima Anandkumar
cs.LG · cs.CE
Quick verdict
PosteriorBench sets a solid bar for distributional benchmarking of generative inverse solvers, though pool-based reference posteriors and author-overlap fairness caveats temper its verdicts.
Summary
PosteriorBench is a benchmark for judging generative inverse-problem solvers as posterior samplers rather than point estimators, targeting the Bayesian formulation p(x|y) ∝ p(y|x)p(x). It spans four physics tasks — Darcy flow inversion, Poisson source recovery, carbon capture and storage (CCS), and light-transport material inference — with reference posteriors built by rejection sampling from large prior pools with Gaussian importance weighting, validated via two independent ensembles (worst-case metric ratios ≤1.13, Table 6). Eight solvers, including five diffusion variants, ES-MDA, ECI, and FNO with MC dropout, are scored with five metrics: posterior-mean and -std errors, MMD, sliced Wasserstein distance, and a radially averaged power-spectrum error. Function-space diffusion samplers (FunDPS, Fun-DDPS, DDIS) perform best on most tasks, yet classical ES-MDA wins on light transport, and MC dropout attains the best spectral error on Darcy, CCS, and LTMI despite poor distributional scores. Ablations show a V-shaped pointwise-distributional tradeoff, out-of-distribution results where wider prior training worsens distributional metrics (Table 3), and guidance weights whose optima are not metric-invariant. Key limitations: references derive from finite prior pools and an LNO surrogate for CCS, all tasks are 2-D synthetic, and several evaluated methods involve the paper's own authors, leaving baseline tuning fairness unstated.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Existing single-ground-truth evaluations are misleading for ill-posed inverse problems: low pointwise error can coexist with severe posterior mismatch (Sec. 4.3, Fig. 4).
- Reference posteriors built by weighted rejection sampling from large prior pools are stable and converged enough to serve as evaluation targets (Sec. C, Table 6 ratios <=1.13).
- Function-space diffusion samplers (FunDPS, Fun-DDPS, DDIS) are the strongest posterior matchers across most tasks, though ES-MDA remains competitive on LTMI (Table 2).
- Guidance weights do not calibrate posterior mean and variance jointly: metric-specific optima diverge even though optima grow with inverse observation-noise variance (Sec. 4.6, Fig. 7).
- Function-space training improves resolution robustness relative to grid-based DiffusionPDE at 128x128 inference (Table 4).
Strengths
Genuine evaluation-protocol shift with validated references
Section 3.2 replaces single-ground-truth scoring with weighted reference posteriors built by rejection sampling and importance weighting; Section C quantifies reference Monte Carlo noise via two independent ensembles, with worst-case solver ratios ≤1.13 across all tasks (Table 6).
Broad, decision-relevant task coverage
Four tasks (Sec. 3.1) deliberately vary prior type (binary GRF, smooth GRF, SGeMS geostatistical, cloud texture), observation pattern (sparse sensors, 16x16 low-res, well columns), and forward physics, covering ambiguity sources the paper argues single-reference benchmarks like InverseBench miss.
Diagnostics go beyond leaderboard numbers
Honest negative and mixed results
Strong reproducibility apparatus
Weaknesses
Reference posteriors inherit prior-pool and surrogate bias
Author overlap with evaluated methods
No inferential statistics
Ambiguity of sample-geometry effect on metrics
2-D synthetic scope
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
PosteriorBench (arXiv 2609.20794) proposes that generative inverse solvers be graded as posterior samplers, not point reconstructors. Four tasks — Darcy flow inversion (binary thresholded-GRF prior), Poisson source recovery (GRFs with per-case alpha in 1.5-3.0 and tau in 2-4), CCS (SGeMS geostatistical priors, ECLIPSE simulation), and light-transport material inference (two-layer media) — are each paired with reference posteriors and scored via five metrics: posterior mean/std errors, MMD, sliced Wasserstein distance, and radially averaged power-spectrum error (Sec. 3.3). Eight solvers are evaluated (Table 2), with ablations on OOD prior coverage (Table 3), guidance versus noise calibration (Fig. 7), and resolution (Table 4). The central claims are that pointwise evaluation misleads (Fig. 4's V-shape), that references are trustworthy (Table 6), and that function-space diffusion samplers lead overall.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Why isn't pointwise reconstruction error enough for evaluating inverse-problem solvers?
Ill-posed inverse problems have many solutions consistent with the same observations. A solver can score well against one reference field while collapsing posterior modes, underestimating uncertainty, or producing an unphysical average. PosteriorBench (arXiv 2609.20794) shows this V-shaped disagreement empirically in its Figure 4 and Section 4.3, motivating distributional metrics instead.
How does PosteriorBench build its ground-truth posteriors?
For each benchmark case, it draws a large pool of prior samples, runs the forward physics, keeps candidates whose observations match the target within a threshold (epsilon = 3 sigma), and assigns Gaussian importance weights. For CCS, a neural-operator surrogate filters roughly 2 million candidates to about 26,000 accepted samples. Two independent reference ensembles validate stability (Table 6).
Which solvers win on the PosteriorBench benchmark?
There is no overall winner. Function-space diffusion samplers — FunDPS, Fun-DDPS, and DDIS — lead on Darcy flow, Poisson, and CCS (e.g., DDIS achieves the best Poisson SWD of 1.8568), but the classical ensemble smoother ES-MDA wins the light-transport task, and MC dropout attains the best spectral error on Darcy, CCS, and light transport despite poor distributional scores (Table 2).
Is the code and data for PosteriorBench available?
The paper states that code is available at github.com/neuraloperator/PosteriorBench, and Appendix E describes the training and evaluation protocol for all eight baselines in enough detail to rerun them, though reference generation requires significant compute (B200 GPUs) and, for CCS, an ECLIPSE-derived dataset.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Reference posterior validation, guidance sweeps, and several result tables live in appendices whose full numeric detail may exceed the extracted text; judgments about them rely on the provided inline tables.
- Figures (1-10) are only available as text fragments; qualitative claims about visualizations (e.g., Fig. 5 error maps) rest on the paper's descriptions.
- The abstract's mention of MCMC is not elaborated anywhere in the extracted text; the actual reference construction is pool truncation plus importance weighting (Sec. B.1).
- Reviewer has not executed the released code; reproducibility assessment is based on documentation completeness in the text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.