Causilo Technical Report
Minyong Cho, Minho Jeong, Dooho Lee, Jinmo Lee, Jaemin Yoo
cs.LG
Quick verdict
Causilo: refine-then-compress tabular TFM claiming frontier Elo at fast inference; strong 3-benchmark evidence, but no ablations or pretraining details.
Summary
Causilo is a tabular foundation model that extends TabICL's column-then-row architecture with a row-refinement module: cell representations exchange information within each row through cross-attention over 4 summary tokens, pass through a second column stage that revisits the context set, and only then are compressed into row embeddings. Because both row stages use fixed summary tokens, feature-side attention stays linear; a microbenchmark measures a feature-scaling exponent of 1.23 versus 1.90-1.99 for TabICLv2, TabFM, Mitra-v2, TabPFN-3.5, and EXAONE-Tabular. The model also bounds QASSMax scaling factors to [1/32, 32] and ensembles over normalization schemes, feature permutations, and class shifts. Pretrained on ~36M synthetic tables (36.08M/37.08M parameters), Causilo reports 1785.4 combined TabArena Elo at a median 0.1043 s per 1K samples, surpassing TabFM's Elo while running ~64x faster; it ranks first on BeyondArena (1346 classification, 1394 regression, 1359 combined) ahead of RealMLP and TabPFN-3, and is Pareto-optimal on ScoringBench's RMSE, R2, MAE, and CRPS at ~0.3 s per fold. Limitations: it trails supervised methods on grouped splits (about 130 Elo behind RealMLP on grouped regression), the extracted text contains no ablations or pretraining details, BeyondArena baselines are published numbers while Causilo was evaluated locally, and the weights are licensed for non-commercial use only.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Causilo achieves 1785.4 combined TabArena Elo at a median 0.1043 s per 1K test samples, exceeding TabFM's Elo at 63.8x its inference speed and lying on the performance-efficiency Pareto frontier.
- Inserting a summary-token row-refinement stage plus a second column stage before row compression restores within-row and cross-row cell interactions while keeping feature-side attention linear in the feature count (measured beta=1.23 versus 1.90-1.99 for five competing TFMs).
- Causilo ranks first on BeyondArena in classification (1346 Elo), regression (1394 Elo), and combined (1359 Elo), ahead of RealMLP and TabPFN-3.
- On ScoringBench, Causilo is Pareto-optimal on RMSE, R2, MAE, and CRPS, outperforming TabPFN-3.5-Fast on all four metrics at approximately 0.3 s per fold.
- Bounded QASSMax keeps attention scaling multipliers within [1/32, 32], preserving coordinate signs and preventing unbounded rescaling at context lengths beyond pretraining.
Strengths
Linear feature scaling verified by a careful microbenchmark
Section 2.2 / Figure 3: fixed 16-row input, features varied 2^4 to 2^16, 20 warm-up runs and 50 CUDA-event timings per model, fit of T(F)=c+aF^beta; Causilo measures beta=1.23 versus 1.90-1.99 for TabICLv2, EXAONE-Tabular, Mitra-v2, TabFM, and TabPFN-3.5, matching the Table 1 complexity analysis.
Consistent frontier results across three independent benchmarks
TabArena combined 1785.4 Elo at 0.1043 s/1K samples (classification 1766.3 at 0.1096 s; regression 2019 at 0.0930 s); first on BeyondArena at 1346/1394/1359 Elo; Pareto-optimal on all four ScoringBench metrics (Section 3.3, Figure 9).
Honest reporting of failure regimes
Architectural specification is complete enough to reimplement
Reproducibility artifacts released
Weaknesses
No ablation studies anywhere in the text
Pretraining is described in one sentence
BeyondArena protocol mixes evaluation contexts
TabArena Elo reported without uncertainty
ScoringBench quantitative values are figure-only
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
Causilo (arXiv 2609.22866, Nums AI, 18 Sep 2026) is an industrial technical report for a tabular foundation model (TFM) targeting the performance-efficiency Pareto frontier. Central claims: (1) a 'refine first, compress later' architecture that restores within-row and cross-row cell interactions on top of TabICL's column-then-row design while keeping feature-side attention linear in the feature count; (2) 1785.4 combined TabArena Elo at a median 0.1043 s per 1K test samples, exceeding TabFM's Elo at 63.8x its inference speed; (3) first place on BeyondArena (1346 classification, 1394 regression, 1359 combined); (4) Pareto-optimality on ScoringBench across RMSE, R2, MAE, and CRPS at about 0.3 s per fold; (5) Bounded QASSMax stabilizing attention scaling within [1/32, 32]. Weights and inference code are released (HuggingFace nums-ai/causilo; code Apache 2.0, weights non-commercial).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Causilo?
Causilo is a tabular foundation model introduced by Nums AI in September 2026. Like TabICL it encodes columns first and compresses rows into embeddings for in-context prediction, but it adds a row-refinement stage and a second column stage before compression, so cell representations exchange information within rows and revisit the labeled context set. It is pretrained on about 36 million synthetic tables and comes in 36.08M-parameter (classification) and 37.08M-parameter (regression) variants.
How fast is Causilo compared to TabPFN?
On TabArena the paper reports a median inference time of 0.1043 seconds per 1,000 test samples at 1785.4 combined Elo. TabPFN-3.5 reaches 1861 combined Elo at 0.4727 seconds (about 4.5x slower than Causilo), while LimiX-2 reaches 1943 Elo at 9.0278 seconds (about 87x slower). The abstract claims 31.6% less inference time than TabPFN-3.5-Fast.
Where does Causilo still lose to traditional ML?
On grouped (non-IID) splits in the BeyondArena benchmark: in grouped classification across 11 datasets it scores 1225 Elo, behind TabM (1283), RealMLP (1282), and LightGBM (1228); in grouped regression across 7 datasets it scores 1238, roughly 130 Elo behind RealMLP (1368) and also below TabM, LightGBM, and CatBoost.
Can I use Causilo commercially?
The inference code is released under Apache 2.0, but the pretrained weights use the Causilo License v1.0, which permits non-commercial research, testing, evaluation, and modification only; commercial, production, or hosted-service use requires a separate license from Nums AI.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures are present only as captions and axis labels; ScoringBench Elo values and several TabArena frontier details are not recoverable from the extracted text.
- No appendix with pretraining, hyperparameter, or ablation content appears in the extraction; such material may exist in the PDF but was not captured.
- Equation typography was partially garbled in extraction (e.g., Eqs. 2, 3, 8); formulations were reconstructed from surrounding prose.
- TabArena numbers reference a living leaderboard snapshot dated 18 September 2026 and will drift.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.