Xiaomi-TabLDM: A Tabular Foundation Model Technical Report
Xiaomi-TabLDM Team, :, Penghui Wang, Wei Liu, Hong Wang, Chengyue Huang, Yuxi Sun, Zirui Wang, et al.
cs.AI
Quick verdict
Claims strong tabular regression via synthetic-pretrained ICL; evidence across 4 benchmarks is solid, but no ablations, overlapping Elo CIs, unquantified TTS.
Summary
Xiaomi-TabLDM is a tabular foundation model (~70M parameters, 61.7-62.7M active) that performs classification and regression by in-context learning, pretrained exclusively on synthetic data generated from structural causal models. Architecture contributions are dual-stream circular-shift feature grouping (one TabICLv2-style dyadic stream, one width-adaptive), lightweight Attention Residual connections at fixed depth strides, and a sparse Mixture-of-Experts in the final eight ICL blocks (2 routed experts, top-1, 1 shared). Training uses a three-stage curriculum over roughly 11-14 days on 8xA100 with Muon. At inference, an optional test-time-scaling pipeline combines feature shuffling, diverse preprocessing views, and NNLS-weighted ensembling. Evaluation spans TALENT (300 datasets), TabArena (51 datasets, 816 tasks), BCCO (156), and OpenML-CTR23 (33): the model ranks 1st on OpenML-CTR23 (average rank 3.03 vs TabFM's 3.06), 2nd in regression on TALENT (4.03 vs TabFM 3.72), TabArena (Elo 1900 vs 2019, with overlapping bootstrap confidence intervals, while using 82% less training time and 68% less prediction time than TabFM), and BCCO (mean rank 2.940), and 4th overall on TabArena (Elo 1659). Limitations: no ablations isolate the proposed components, the test-time-scaling gain over the base model is never quantified, Elo intervals on the 13-dataset regression subset overlap broadly, and Table 3's BCCO regression metrics are suspiciously identical across Accuracy/AUC/LogLoss.
Review complete
Assessment at a glance
6
risk flags identified
● high 1
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Xiaomi-TabLDM achieves the best average rank on OpenML-CTR23 (3.03) and second place in regression on TALENT, TabArena, and BCCO, behind only TabFM.
- On TabArena regression it reaches the second-highest Elo (1900) while using 82% less training time and 68% less prediction time than TabFM (6.99s/3.12s vs 38.85s/9.67s per 1K).
- Exclusively synthetic SCM pretraining combined with dual-stream feature grouping, Attention Residual connections, and sparse MoE yields richer feature interactions and efficient capacity scaling.
- Test-time compute scaling (feature shuffling, diverse preprocessing views, NNLS-based combination) consistently improves predictive performance over the base model.
- In a 2,240-dataset synthetic noise study, the model attains the highest R2 on 70.3% of datasets and stays robust across twelve noise families, SNR levels, and graph densities.
Strengths
Consistent regression dominance across four complementary benchmark suites
Ranks 1st on OpenML-CTR23 (avg rank 3.03 over 33 datasets, Sec. 4.4), 2nd on TALENT regression (4.03/100 datasets, Sec. 4.1.1), 2nd on TabArena regression (Elo 1900, Table 2), and 2nd on BCCO regression (2.940, Table 3) - a cross-suite pattern that is hard to attribute to benchmark-specific tuning.
Genuine efficiency result at default configuration
Table 2: second-highest regression Elo (1900) with 6.99s train / 3.12s predict per 1K samples vs TabFM's 38.85s / 9.67s (82%/68% less), beating tuned+ensembled pipelines like AutoGluon 1.5 extreme (1776) and RealTabPFN-2.5 T+E (1736).
Unusually transparent appendix for an industry report
Methodologically honest sensitivity analysis
Weaknesses
Zero ablation evidence for the three claimed architectural contributions
Test-time scaling (Sec. 3) is described but never quantified
Statistically fragile headline rankings
Table 3 anomaly in the BCCO regression block
Classification strength is narrower than the framing suggests
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is an industry technical report introducing Xiaomi-TabLDM, a ~70M-parameter tabular foundation model (~61.7-62.7M active, Table 10) that performs classification and regression via in-context learning from purely synthetic SCM-generated pretraining data. Three claims anchor the paper: (i) a new regression performance standard - 1st on OpenML-CTR23, 2nd on regression across TALENT, TabArena, and BCCO (Sec. 1, Sec. 4); (ii) a favorable efficiency trade-off - second-highest TabArena regression Elo (1900) with 82% less training and 68% less prediction time than TabFM (Table 2); (iii) three architectural contributions (dual-stream grouping, Attention Residual, sparse MoE, Sec. 2.1) plus a test-time-scaling framework (Sec. 3). The evaluation scope is broad: four public suites (TALENT 300 datasets, TabArena 51/816 tasks, BCCO 156, OpenML-CTR23 33) and a 2,240-dataset synthetic noise-robustness factorial (Appendix C).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Xiaomi-TabLDM?
A tabular foundation model from Xiaomi (~70M parameters) that performs classification and regression via in-context learning: it is pretrained only on synthetic datasets generated from structural causal models, then predicts on new datasets from labeled examples without any task-specific training.
How does Xiaomi-TabLDM compare to TabPFN and TabFM?
On TabArena's 13-dataset regression subset it scores Elo 1900 - second to Google's TabFM (2019) but ahead of TabPFN-3 (1800) - with 82% less training and 68% less prediction time than TabFM. On the OpenML-CTR23 regression benchmark it has the best average rank (3.03 vs TabFM's 3.06). Its classification results are competitive but not top-ranked overall.
Is Xiaomi-TabLDM code and model weights available?
The report lists a GitHub repository (github.com/xiaomi-research/xiaomi-tabldm) and a Hugging Face page (huggingface.co/occams/Xiaomi-TabLDM); this review could not verify their current contents or licenses.
What is architecturally new in Xiaomi-TabLDM?
Three components: dual-stream circular-shift feature grouping (one fixed dyadic stream, one width-adaptive), lightweight Attention Residual connections that learn to fuse earlier block outputs, and a sparse Mixture-of-Experts in the final eight in-context-learning blocks, trained with a three-stage curriculum on synthetic SCM data.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The prompt's abstract block is cut mid-sentence ('where alloc'), but the complete sentence appears in the body text (Sec. 1), so no content is actually missing from the extraction.
- Figure contents (e.g., Figures 1-3, 6-9, 10-11) were extracted as unordered vector text; value-to-model assignment in plots is inferred from the accompanying rank lists and could be misaligned for individual bars.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.