TabBench-Bio: A Living Benchmark for Machine Learning on High-Dimensional Biomedical Tables
Jules Kreuer, Sofiane Ouaari, Julia Hellmig, Julius Braitinger, Nico Pfeifer
cs.LG · cs.AI
Quick verdict
TabBench-Bio benchmarks 19 models on 43 biomedical HDLSS tables; RealTabPFN v2.5 leads (Elo 1274) — rigorous, honest, but metric-fragile headline ranking.
Summary
TabBench-Bio is a living benchmark of 43 curated biomedical datasets (30 classification, 13 regression) spanning seven modality groups, evaluating 19 model configurations plus a constant baseline and a separately reported one-hour AutoGluon reference under a shared 4x7 grid of feature caps and training-sample budgets with common cross-validation folds. At the reference cell of 10,000 features and 100 training samples (39 targets), RealTabPFN v2.5 tops the Bradley-Terry ranking at Elo 1274, 145 Elo ahead of Logistic Regression with a paired-bootstrap interval excluding zero, while the raw target-level win score is not significant. The strongest model varies with operating point and biomedical modality. Strengths include deterministic reproducibility artifacts, explicit leakage controls, and unusually candid statistical reporting; weaknesses include metric sensitivity of the headline ranking, model-based Elo assumptions, possible cohort overlap among targets, and small per-modality pools.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 4
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- TabBench-Bio provides a living, versioned benchmark of 43 biomedical datasets evaluated under one shared 4x7 feature-by-sample grid protocol with reusable fold-level artifacts.
- At the reference cell (10,000 features, 100 training samples), RealTabPFN v2.5 ranks first at Elo 1274, separated from Logistic Regression by 145 Elo (95% CI [59, 232]) under a paired target-level bootstrap.
- The strongest model depends on the operating point and biomedical modality, so a single global leaderboard cannot guide model selection.
- Operating beyond a declared feature limit affects models very differently: TabICL loses 205 Elo and 12 rank positions from 2,000 to 10,000 features while RealTabPFN v2.5 loses 2 Elo.
- Fixed-default comparisons - not tuned or ensembled variants - are the right target for measuring reusable out-of-the-box reliability in biomedical HDLSS settings.
Strengths
Feature-by-sample grid as a primary design, not an ablation
The 4x7 grid with shared folds and nested feature caps yields findings single-point benchmarks cannot: widening features 2,000 to 10,000 at n=100 drops TabICL from rank 4 to 16 (-205 Elo) while RealTabPFN v2.5, under the same declared 2,000-feature limit, holds rank 1 (-2 Elo) (Section 4.2).
Reproducibility infrastructure is unusually complete
Fold-level predictions, run status, deterministic aggregation manifests with cryptographic digests, a versioned registry, GitHub code, and a live leaderboard (Section 3.5, Section 8) make every reported number auditable.
Leakage controls are explicit and consistently applied
Candid statistical self-reporting
Scope and curation breadth
Weaknesses
The headline podium is metric-dependent
Statistical separation is fragile
Failures are imputed inside the ranking
Target independence assumption is strained
Modality-level evidence is thin and partly incomparable
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
TabBench-Bio (arXiv 2609.07441, University of Tübingen) is a benchmark/resource paper for machine learning on high-dimensional, low-sample-size (HDLSS) biomedical tables. It contributes a versioned registry of 43 datasets (30 classification, 13 regression; seven modality groups including gene expression, metagenomics, DNA methylation, molecular properties, genomic prediction, and protein/genomic-language-model embeddings), 19 model configurations plus a constant baseline, and a leak-resistant 4×7 grid of feature caps (2,000/10,000/25,000/full) × training-sample budgets (20-1,000/full) under shared 5-fold cross-validation. AutoGluon's one-hour "extreme" preset is deliberately excluded from the peer ranking and reported as a resource-intensive reference (Elo 1124 at the reference cell, Figure 2). Headline claims: (1) at the prespecified reference cell (p=10,000, n=100; 39 targets), RealTabPFN v2.5 leads at Elo 1274 [1187, 1366], 145 Elo above Logistic Regression ([59, 232]); (2) the strongest configuration depends on operating point and modality; (3) grid-based evaluation exposes behaviors single-point benchmarks miss.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is TabBench-Bio?
A living, openly maintained benchmark for machine learning on high-dimensional, low-sample-size biomedical tables. Its first release covers 43 curated datasets (30 classification, 13 regression) from sources such as OpenML, MGnify, TCGA, GEO, TDC and ChEMBL, evaluated with 19 model configurations under a shared cross-validation protocol across a grid of feature counts and training-sample budgets. Code is on GitHub and an interactive leaderboard runs at tabbench-bio.eu.
Which model performs best on TabBench-Bio?
At the reference operating point (10,000 features, 100 training samples, 39 targets), RealTabPFN v2.5 has the highest Bradley-Terry Elo (1274), 145 Elo above Logistic Regression with a paired bootstrap interval of [59, 232]. Tabular foundation models generally hold the leading ranks, but the best model depends on the feature/sample budget and the biomedical modality; for example, Extra Trees and Random Forest lead the four genomic-prediction regression targets.
Is RealTabPFN v2.5 significantly better than Logistic Regression in this benchmark?
By the paper's paired target-level bootstrap, yes: the 145-Elo difference has a 95% interval of [59, 232], excluding zero. However, the model-free target-level win score is 57.7% (95% interval [43.6%, 71.8%], which includes 50%), and under AUROC instead of macro-F1 Logistic Regression drops from rank 2 to rank 7. So the separation is significant within the Bradley-Terry model and the chosen metric, but it is not overwhelming.
What are the main limitations of TabBench-Bio?
The registry is classification-heavy and dominated by gene-expression and metagenomics tasks; feature caps are random and label-agnostic (five seeded orderings); all models run with fixed defaults rather than tuned configurations; sample-budget contrasts change both the number and the identity of training rows; related targets may share cohorts or platforms, which the target-level bootstrap does not model; and classes with fewer than ten samples are removed. Modality-level rankings rest on 3-10 targets each and are descriptive.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The supplied abstract copy ends mid-sentence ('The interactive lead'), but the body text restates the full sentence and the leaderboard URL, so no content is lost.
- PDF extraction renders some diacritics as spacing artifacts (e.g., 'M ¨uller'); this does not impair the review.
- Figures 2 and 3 are available only through axis labels and embedded label text in the extraction; all cited values (e.g., AutoGluon Elo 1124, Pareto frontier) were legible in that text.
- No truncation at the far end of the provided text: the registry table (Appendix C) and the local-dataset construction details (C.1) are fully present.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.