A Scaling Study for fMRI Foundation Models
Wenhao Ye, Xuanye Pan, Junfeng Xia, Junxiang Zhang, Mo Wang, Quanying Liu
cs.LG
Quick verdict
Controlled study: fMRI scaling can't be read from compute alone—data and model size must co-scale. Rigorous and honest, but bound to one pretraining method.
Summary
The authors run a controlled empirical scaling study for fMRI foundation models. Holding the pretraining objective fixed (NeuroJEPA, a LeJEPA-style joint-embedding method on Schaefer-100 ROI-time series), they vary pretraining data across six strictly nested corpora (2,585-63,401 recordings from over 200 source datasets), encoder size from 2.65M to 92.59M parameters, and training progress, evaluating frozen linear probes on 12 downstream tasks (~10,347 GPU-hours). Three patterns emerge: performance generally improves with compute, but configurations at similar compute diverge; the fitted benefit of additional data grows with model size (beta D×N rises from 0.123 at schedule midpoint to 0.239 at completion); and at matched compute, data scaling helps 8 of 12 tasks while model scaling helps none. They then fit an effective-data response on five in-distribution tasks, select two models at locked compute budgets (0.128 and 1.97 EFLOP, yielding 2.65M and 10.42M models), and report that the 10.42M model attains the highest average out-of-distribution performance among six public fMRI baselines using far less pretraining compute. Main limitations: conclusions are bound to one method and the observed grid; baseline compute figures are reconstructed estimates; OOD datasets appear in descriptive analyses; and clinical task accuracies remain modest.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 4
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Downstream performance generally improves with pretraining compute, but compute alone does not characterize fMRI scaling: configurations with similar estimated EFLOPs can differ substantially (§5.1, Fig. 3a).
- Larger models show larger fitted gains from additional pretraining data, and this interaction strengthens late in training (βD×N: 0.123 at p=0.5 to 0.239 at p=1.0; Δβ=0.116, 95% CI [0.006, 0.216]).
- At matched compute, data scaling is the more consistent lever: 8 of 12 tasks favor more data, 4 are unresolved, and none clearly favors a larger model (§5.3, Table 11).
- ID-guided selection at two locked budgets (0.128 and 1.97 EFLOP) yields NeuroJEPA models of 2.65M and 10.42M parameters that achieve the highest average OOD performance among the compared public fMRI foundation models (§6, Tables 2 and 13).
Strengths
Genuinely controlled design
Six strictly nested, subject-grouped pretraining corpora preserve source proportions so the data axis varies quantity without confounding composition (§4.1); objective, architecture family, and downstream protocol are held fixed, cleanly isolating the scaling variables.
Locked selection protocol with honest audit
The ID response is fitted on five in-distribution tasks and both models locked before any OOD evaluation (§6); the archived audit openly reports that the selected models rank only 8/35 and 24/42 among eligible candidates with regrets 0.0211/0.0359 (Table 14) — candor rarely seen in this literature.
Above-average uncertainty treatment
Transparent compute accounting
Self-aware scoping of claims
Weaknesses
Single-method scope
Baseline compute comparability is approximate
Modest effect sizes, no multiplicity control
No simple non-deep baseline
Code link is a placeholder
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Review: A Scaling Study for fMRI Foundation Models
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What are the main findings of this fMRI scaling study?
The paper finds that downstream performance of fMRI foundation models generally improves with more pretraining compute, but compute alone does not determine results — configurations spending similar compute can differ substantially. Larger models benefit more from additional pretraining data, and at equal compute, scaling data helps more tasks (8 of 12) than scaling model size (0 of 12). Two models selected using only in-distribution performance at fixed compute budgets (0.128 and 1.97 EFLOP) achieved the highest average out-of-distribution performance among the compared public fMRI foundation models.
What is NeuroJEPA?
NeuroJEPA is the fixed pretraining method used throughout the study: a LeJEPA-style joint-embedding approach applied to 100-parcel (Schaefer) ROI-by-time fMRI segments. It aligns two global and six local augmented views through a shared encoder and uses sketched isotropic Gaussian regularization (SIGReg) to keep embeddings well-spread; it uses no EMA teacher network, no masked-reconstruction head, and no training curriculum.
How much compute did the study use?
Approximately 10,347 GPU-hours (about 9,680 on NVIDIA A800-80GB), with pretraining accounting for roughly 91.5%. Per-model pretraining compute is reported in EFLOPs: the two final selected models used 0.128 and 1.97 EFLOP, compared with 0.129–206.77 EFLOP for the public baseline models they evaluate.
Are the code and data available for this paper?
The paper states that code is available (the extracted preprint shows a placeholder link), and Appendix C catalogues all 201 curated fMRI source datasets with participant and recording counts. The downstream datasets (ABIDE, HCP, PNC, PPMI, ADNI, ADHD-200, NKI-RS, BHRC, SALD) are public, and baseline models are evaluated from their official released weights.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–7 are not analyzable from the extracted text; quantitative claims rely on in-text numbers and appendix tables.
- Two passages were truncated in extraction (part of the §5 opening and part of the appendix references); ADNI-MCI results are referenced but not shown.
- The code URL appears as a literal placeholder ('Code is available at link'); availability could not be verified.
- Baseline EFLOP values are the paper's own reconstructions and could not be independently verified in this review.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.