Benchmarking Optimizers to Solve Inverse Problems with Differentiable Physics Simulators
cs.LG · math.OC · physics.comp-ph
Quick verdict
Claims first optimizer benchmark for 12 differentiable-physics inverse problems; useful unified PyTorch suite but 3-seed stats and toy scale limit conclusions.
Summary
This paper presents what the authors describe as the first systematic benchmark of optimizers for inverse problems solved with differentiable physics simulators. The authors implement 12 PyTorch simulators spanning discrete mechanics, continuous mechanics, atomistic simulation, rendering, and semi-empirical models, and define 12 inverse problems in three classes: parameter identification, inverse design, and optimal control. They compare 12 optimizers — SGD, RMSprop, Adam, AdamW, L-BFGS, K-FAC, Shampoo, SOAP, random search, simulated annealing, particle swarm, and differential evolution — on an A100 GPU, using a shared 207-configuration hyperparameter grid with the best configuration chosen by mean best loss over three random-seed repeats of 100 epochs. Headline findings: L-BFGS dominates structured spectral fitting (tight-binding loss ~1.1e-9 versus ~3.2e-5 for SGD); global optimizers excel mainly on low-dimensional tasks; and global steps cost roughly 5–19 s versus ~1 s for local methods on DFT fitting. The SIMP topology column excludes non-converged negative-loss runs (~3%). Limitations include tiny toy problems (1–1100 variables), only three seeds with no significance testing, hyperparameter selection on the reported runs, and no stated code release, which limits reproducibility of the benchmark as published.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- This is claimed to be the first comprehensive benchmark of optimizers for inverse problems solved with differentiable physics simulators (Sec 1, Sec 5).
- Approximate second-order methods decisively help on structured spectral fitting: L-BFGS reaches 1.1e-9 on tight-binding fitting versus 3.2e-5 for SGD (Table 4).
- Global optimizers improve robustness mainly in low-dimensional search spaces or noisy landscapes, not automatically because a model is physics-based (projectile zeros vs SA/DE plateau at 2.5-2.6 on 600-variable topology optimization, Table 5).
- Global-optimizer steps cost roughly an order of magnitude more wall time than local steps (5-19 s versus ~1 s per step on DFT fitting, Table 8).
Strengths
Genuine gap, unified artifact
First attempt to compare 12 optimizers across 12 differentiable-physics tasks in one PyTorch framework (Sec 3); DiffTaichi and PhiFlow are named in Sec 1 as the closest prior works but are explicitly not optimizer benchmarks.
Simulators are mathematically specified
Appendix A gives full discretizations — mass-spring impulse rules (Eq. 7–9), rigid-body contact/friction impulses (Eq. 16–17), Q4-FEM with the SIMP law (Eq. 20–24), Brinkman–Stokes with artificial compressibility, and the TB Bloch Hamiltonian (Eq. 26) — enabling independent reimplementation.
Transparent failure filtering
Compute cost is reported, not ignored
Analysis extracts actionable lessons
Weaknesses
Statistical reporting is thin
Post-hoc hyperparameter selection biases family comparisons
Budget asymmetry across optimizers
Toy scale and fidelity with an unsupported extrapolation claim
No stated code or data release
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper claims, in its abstract and Section 5, to be the first benchmark for optimizers solving inverse problems with differentiable physics simulators. The artifact is a zoo of 12 PyTorch simulators across five domains (Table 1): discrete mechanics (projectile, flight dynamics, mass-spring, rigid-body), continuous mechanics (2D linear-elastic FEM, 2D Brinkman-Stokes wind tunnel), atomistic simulation (a radial helium-like Kohn-Sham DFT, a spinless 1D tight-binding ring), rendering (orderless 2D Gaussian splatting, single-scattering atmosphere), and semi-empirical models (Pacejka magic-formula tire, first-order Thevenin battery). On these it defines 12 inverse problems (Table 2) in three classes — 5 parameter-identification, 4 inverse-design, 3 optimal-control — with design dimensions from 1 (launch angle) to 1100 (mass-spring open-loop table). Twelve optimizers in three families are compared. Central claims: (i) curvature helps selectively, most decisively on structured spectral fitting; (ii) global optimizers pay off only in low-dimensional or noisy settings; (iii) compute trade-offs favor first-order methods for inner-loop use. These are broadly supported within the benchmark's own scope, but the priority claim is asserted rather than systematically substantiated — DiffTaichi and PhiFlow are discussed in Section 1 only as non-benchmark neighbors.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Which optimizer is best for differentiable physics simulation?
According to this benchmark of 12 optimizers on 12 differentiable-physics inverse problems, no single optimizer wins everywhere. L-BFGS is strongest on smooth structured fitting tasks (tight-binding spectral fitting at ~1.1e-9 loss vs ~3.2e-5 for SGD); adaptive first-order methods are reliable generalists, with Shampoo and RMSprop leading on the mass-spring control task; global optimizers (random search, simulated annealing, particle swarm, differential evolution) mainly pay off on low-dimensional problems and can cost roughly 5-19 s versus ~1 s per step on some tasks.
Is the code for this optimizer benchmark available?
The paper text does not include any code or repository link. It does, however, give detailed mathematical specifications for all 12 simulators in Appendix A and the full hyperparameter search space in Appendix B (207 configurations), so the benchmark could in principle be reimplemented independently.
What problems and simulators does the benchmark cover?
Twelve PyTorch simulators across five domains — discrete mechanics (projectile motion, flight dynamics, mass-spring, rigid-body), continuous mechanics (elastic FEM, fluid drag in a wind tunnel), atomistic simulation (radial two-electron DFT, tight-binding ring), rendering (Gaussian splatting, atmospheric scattering), and semi-empirical models (Pacejka tire magic formula, Thevenin battery) — with 12 inverse problems in parameter identification, inverse design, and optimal control, ranging from 1 to 1100 optimization variables.
How reliable are the benchmark's conclusions?
The directional conclusions (curvature helps structured spectral fits; global methods mainly help in low dimensions; global steps are ~10x costlier) are consistent across the result tables, but each number averages only 3 random seeds with no statistical tests, hyperparameters were tuned on the same runs reported, and task scale is small (1-1100 variables). Fine-grained rankings, especially differences within one order of magnitude, should be treated as indicative rather than definitive.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1 is referenced as a gallery of the 12 simulators, but image content is not present in the extracted text, so the visual sanity of the simulators could not be assessed.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.