Inductive Biases in Field-Level Cosmological Inference from Galaxy Catalogs
James O. Baldwin, Shy Genel, Francisco Villaescusa-Navarro
astro-ph.CO · cs.LG
Quick verdict
Claims Deep Sets infer Ωm from velocities alone (~18-25% err), GNNs exploit positions (~10%); plausible but single-run, with rival-code discrepancy unresolved.
Summary
The paper asks how architectural inductive biases govern cosmological parameter inference from simulated galaxy catalogs. Using CAMELS hydrodynamic simulations (25 h⁻¹Mpc boxes; Astrid, IllustrisTNG LH and SB28, Simba, Swift-EAGLE), the authors train supervised neural estimators predicting Ωm (and σ8) with heteroscedastic uncertainty, comparing permutation-invariant Deep Sets built on MLPs or Kolmogorov-Arnold Networks (KANs) against graph neural networks (GNNs), with galaxy positions, line-of-sight peculiar velocities, or both as inputs. Trained on Astrid with mass-cut marginalization, velocity-only Deep Sets recover Ωm with ~17-18% mean relative error in-distribution and ~24-26% out-of-distribution, with KANs and MLPs equivalent; positions alone yield no extractable signal and adding positions does not consistently help Deep Sets. GNNs using positions plus velocities improve to roughly 10% in-distribution and 10-17% OOD error. σ8 is not recovered by velocity-only set models, which the authors attribute to small box volume. Coverage diagnostics show in-distribution calibration (f1σ=0.656) degrading to overconfidence under cross-suite shift (f1σ=0.491). Limitations: exact simulated velocities (no noise, RSD, survey geometry), single-run metrics without seed variance, an unresolved implementation discrepancy versus de Santi et al. 2023's PyG MetaLayer code (Appendix B), and no tabulated GNN numbers.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Deep Sets infer Ωm from line-of-sight peculiar velocities alone with ~17-18% mean relative error in-distribution (Astrid) and ~24-26% across other CAMELS suites, with KAN and MLP implementations performing equivalently.
- Positional information yields no extractable Ωm signal in permutation-invariant set architectures, and adding positions to velocities does not consistently improve them.
- GNNs with explicit galaxy-galaxy edges exploit positional information, improving Ωm inference to ~10% in-distribution and 10-17% cross-suite mean relative error.
- σ8 is not recoverable from velocity-only set-based models in the (25 h⁻¹Mpc)³ CAMELS setting, plausibly because the box samples too few independent 8 h⁻¹Mpc-scale regions.
- Predicted heteroscedastic uncertainties are approximately calibrated in-distribution (f1σ=0.656) but become overconfident under cross-suite shift (f1σ=0.491).
Strengths
Simple baselines establish signal specificity
Table 2 shows a training-mean predictor (ε 36-48%) and a log N linear regression (ε 36-44%) are clearly dominated by trained velocity models (ε ~17-26%), ruling out prior- or abundance-only shortcuts.
Unusually careful scoping of claims
Section 1 disambiguates field-level usage from Bayesian forward modeling and SBI; the OOD definition (Section 3) is explicitly restricted to cross-suite transfer, and the paper enumerates every shift it does NOT test (cosmology ranges, RSD, geometry, velocity noise).
Uncertainty diagnostics with honest negative findings
Reproduction attempt with disclosed discrepancy
Replication across training suites
Weaknesses
Single-run metrics with no variance estimates
Headline result is implementation-dependent
Position encoding may predetermine the negative result
GNN results lack tabulated numbers
KAN variance miscalibration signals undiscussed
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks which galaxy observables and neural architectures carry extractable Ωm information, holding the inference task fixed. It compares permutation-invariant Deep Sets (MLP-based and KAN-based 'DeepKAN') against message-passing GNNs, using line-of-sight peculiar velocities (vz) and transformed positions, on CAMELS hydrodynamic catalogs. Scoping is unusually careful: Section 1 disambiguates 'field-level' from Bayesian forward-model and simulation-based-inference usages; the OOD definition (Section 3) is explicitly restricted to cross-suite transfer, with the untested shifts (cosmology ranges, redshift, volume, geometry, RSD, velocity noise) enumerated. Central claims: (1) velocity-only Deep Sets recover Ωm (~17-18% ID / ~24-26% OOD mean relative error, Table 3) with KAN ≈ MLP; (2) positions add nothing to set models and carry no standalone signal; (3) GNNs exploit position relationally (~10% ID / 10-17% OOD); (4) σ8 is not recoverable; (5) uncertainties become overconfident under shift.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Can machine learning infer cosmology from galaxy velocities alone?
In this CAMELS-simulation study, yes for Ωm: permutation-invariant Deep Sets trained only on line-of-sight peculiar velocities recover Ωm with ~17-18% mean relative error in-distribution and ~24-26% when tested on simulations using different subgrid physics, clearly beating count-based baselines (~36-48% error). σ8, however, was not recoverable in the same setting.
Do KANs beat MLPs for cosmological inference?
Not in this paper. Kolmogorov-Arnold Networks (KANs) and conventional MLPs perform equivalently inside identical Deep Set architectures for Ωm inference from velocities, which the authors read as evidence that performance is limited by information content and architectural inductive bias rather than by the flexibility of univariate function approximation.
Why can't the models constrain σ8 from peculiar velocities?
The authors argue that the (25 h⁻¹Mpc)³ CAMELS boxes sample only a few independent 8 h⁻¹Mpc-scale regions and omit the long-wavelength modes that drive coherent velocity flows, so fluctuation-amplitude information is not recoverable by the learned summaries — consistent with earlier GNN and galaxy-catalog studies that also failed to recover σ8.
Are these results applicable to real survey data?
Not directly. The models consume exact simulated peculiar velocities, whereas surveys infer velocities from noisy distance indicators (Tully-Fisher, Fundamental Plane, Faber-Jackson, Type Ia supernovae) with selection effects and incomplete sky coverage. The authors explicitly state that validation under velocity-measurement noise, selection effects, and survey geometry is required before survey application.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure panels (scatter plots in Figures 1-8) are images not rendered in the text extraction; quantitative reading of GNN and σ8 results relies on captions, the abstract, and in-text numbers rather than plotted points.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.