Do Tabular Foundation Models Know Physics? Contamination, Units, and the Deterministic Limit
Wassim Tenachi, Yashar Hezaveh, Laurence Perreault Levasseur, Pierre-Luc Bacon
cs.LG · astro-ph.IM
Quick verdict
Claims TFMs' priors can't represent noiseless laws or units, so they interpolate but don't model physics; rigorous study, statistically thin.
Summary
The paper asks whether tabular foundation models (TFMs) have learned physics, reframing the question via their Bayesian construction: what does the pretraining prior contain? Four open-weight TFMs (TabPFN-3, TabICLv2, TabDPT, Real-TabPFN-2.5) are evaluated against six trained baselines (RealMLP, TabM, MLP, CatBoost, GP, Ridge) on tables regenerated from 316 physical equations (Feynman 120, LSR-Transform 111, LSR-Synth 85), sweeping nsamples {50, 200, 2000}, multiplicative noise sigma {0, 0.01, 0.1}, in-domain (k=1) versus k=2 shell extrapolation, and 3 seeds, scored by NMSE mean rank per stratum. A corpus audit plus a seen-versus-unseen exposure experiment finds no detectable contamination, with an admitted detection bound of roughly one rank position. TFMs lead in every stratum and survive tuning and ensembling, the sole exception being the GP beating the cheapest TFM setting at the largest context. Two structural limits are demonstrated: at sigma=0 predictive widths plateau (0.010-0.020 normalized in App. A) while error keeps falling, and error tracks raw column count rather than the number of Buckingham pi groups, penalizing TabPFN-3 while benefiting RealMLP and CatBoost. Limitations: three seeds with no reported significance tests, an underpowered contamination null, a single extrapolation distance, a disclosed 20% validation carve-out favoring TFMs, and no code release stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- TFMs pretrained without any physical law dominate six trained baselines on 316 physics-equation datasets across context-size, noise, split, and dimension strata, and tuning plus ensembling does not reverse this (sole exception: GP over the cheapest TFM setting at the largest context).
- A corpus audit plus a seen-vs-unseen exposure experiment finds no detectable contamination of the real-data pretraining corpora by physics-equation data, with detectability bounded at roughly one rank position.
- TFM priors cannot represent a noiseless mechanism: at sigma=0 the predictive width plateaus (0.010-0.020 normalized, App. A) while error keeps decreasing, whereas at sigma=0.1 the models are correctly calibrated.
- TFM priors are unit-blind: error tracks raw column count rather than the number of Buckingham pi groups, and dimensional structure penalizes TabPFN-3 while benefiting RealMLP and CatBoost.
Strengths
Two-layer contamination audit that measures rather than asserts
Sec. 3 combines a corpus audit (App. E enumerates TabDPT's 122 OpenML datasets and Real-TabPFN-2.5's 43; no Feynman equations, no analytic regression targets, physics only as classification tasks like HIGGS/MAGIC/MiniBooNE) with a seen-vs-unseen exposure experiment against unexposed anchors (Fig. 2), correctly noting that name/hash filtering would miss a renamed resample. The null is reported with an explicit power bound (~one rank position).
Mechanistic diagnostics beyond leaderboard scores
Fig. 4a shows the deterministic-limit failure: at sigma=0 predictive width plateaus while RMSE keeps falling, yet at sigma=0.1 the same models sit at the expected calibration width, isolating the noiseless case. Fig. 4b introduces a genuinely novel probe via n_pi = n_vars - rank(D) (Buckingham pi), showing error tracks column count, not dimensionless dimensionality.
Protocol hygiene in App. C
Unusually honest self-refutation of suggestive results
Weaknesses
Significance language without visible statistics
Underpowered contamination null by the authors' own admission
Extrapolation characterized at a single distance
Disclosed but unresolved budget asymmetry
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks a sharply framed question: what do tabular foundation models' (TFM) Bayesian priors contain with respect to physics? Building on the posterior-predictive reading of TFM pretraining (Hollmann et al., 2025), it probes four open-weight TFMs (TabPFN-3, TabICLv2, TabDPT, Real-TabPFN-2.5) against six trained baselines (RealMLP, TabM, MLP, CatBoost, GP, Ridge) on tables regenerated from 316 equations (Feynman 120; LSR-Transform 111; LSR-Synth 85). Three claims carry the paper: (1) TFMs lead in every stratum and survive tuning/ensembling; (2) contamination is undetectable under a two-layer audit; (3) the prior can represent neither a noiseless mechanism nor dimensional structure. The prior-probing framing is the paper's best idea — law-sampled data is the cleanest instrument for asking what a prior contains — and the conclusions are appropriately scoped ('interpolate physics without yet acting as physical models').
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Do tabular foundation models like TabPFN understand physics?
According to this paper, they interpolate physics data extremely well despite never being pretrained on physical laws, but their Bayesian prior lacks two properties every physical law has: it cannot represent a noiseless mechanism (they always report nonzero uncertainty even on perfectly deterministic data), and it is blind to units/dimensional structure (their error tracks the number of columns rather than the number of dimensionless groups a law actually depends on). The authors' conclusion: excellent amortized interpolators, not yet physical models.
Was TabPFN contaminated by the Feynman datasets it was evaluated on?
The authors audited the enumerated pretraining corpora of the two real-data models: TabDPT's 122 OpenML datasets and Real-TabPFN-2.5's 43 datasets contain no Feynman equation and no regression target generated by an analytic function; physics appears only as classification tasks (e.g., HIGGS, MAGIC, MiniBooNE). A seen-versus-unseen exposure experiment also came back null, though the paper notes any effect smaller than about one rank position would have been invisible at these pool sizes.
Can TabPFN extrapolate beyond its training data on physical systems?
Largely no. On a test shell at twice the training box scale, the field compresses and rankings reshuffle. In the paper's damped-oscillator demo, all four TFMs relaxed to a constant within roughly half a period beyond the sampled range; TabICLv2's oscillatory-looking uncertainty banding was shown by a phase cross-correlation test not to be phase-locked to the true oscillation, so it is multimodal hedging rather than a recovered frequency.
What do 'contamination', 'units', and 'the deterministic limit' in the title refer to?
Contamination: whether the models secretly saw the benchmark equations during pretraining (audited; no evidence found). The deterministic limit: on noiseless data the correct prediction is a single point, but distributional TFMs report a predictive width that stops shrinking (~0.010-0.020 normalized) no matter how much context arrives. Units: physics variables carry SI dimensions, so a five-variable law may depend on only two dimensionless groups (Buckingham pi); unit-blind, permutation-invariant TFMs cannot exploit this and the strongest one is actually penalized by it.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Values plotted inside figures (Figs. 1-3, 5-6 curves) are not present in the extracted text; plot-dependent judgments rely on the numeric values quoted in captions and body text (e.g., width floor 0.010-0.020, footnote 6 ranks).
- PDF extraction garbles some glyphs in figure labels (e.g., sigma rendered as '¾'), but all formulas and numbers cited in this review were reconstructable and verified against the surrounding text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.