Do Center Biases Propagate? Robustness of Pathology Foundation Models in Whole-Slide Image Classification
Ilán Carretero, Pablo Meseguer, Rocío del Amor, Valery Naranjo
cs.CV · cs.AI
Quick verdict
Claims center-encoded bias in PFMs propagates to WSI predictions; sound Cramér's V design, but no variance or significance reporting weakens the conclusions.
Summary
This conference proceedings paper asks whether acquisition-center information encoded by pathology foundation models (PFMs) leaks into weakly supervised whole-slide-image (WSI) classification. The authors construct controlled training sets whose class–center association is tuned to Cramér's V ∈ {0, 0.25, 0.5, 0.75, 1} while the test set stays uncorrelated, then benchmark six frozen PFMs (CONCH, KEEP, VIRCHOW2, H-Optimus-1, KAIKO, UNI-2) with two MIL aggregators (ABMIL, TransMIL) on four binary tasks (AI4SKIN, CAMELYON16, TCGA-BRCA, TCGA-NSCLC), using 5-fold cross-validation and 10 random samplings per configuration. They propose AUCC, the trapezoidal area under balanced accuracy as a function of V, to jointly summarize performance and robustness, and test ComBat embedding harmonization as a fix. Results: BACC degrades consistently as correlation grows; ABMIL yields higher average AUCC than TransMIL for five of six PFMs (+1.18 pp mean); the vision-language models CONCH and KEEP show the most center-robust representations on FM-SI/RI metrics, while KAIKO encodes center information most strongly; ComBat helps in only 14 of 48 configurations (mean −0.63 pp), with gains concentrated on CAMELYON16 and near-systematic losses on TCGA-BRCA. Limitations: two acquisition centers per dataset, binary tasks only, no variance or significance estimates despite repeated sampling, and no code release.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 16 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Center-related information encoded in PFM patch representations propagates to slide-level predictions, with balanced accuracy degrading as the induced class–center correlation (Cramér's V) increases.
- The MIL aggregator is itself a robustness factor: ABMIL achieves higher average AUCC than TransMIL for five of six PFMs, by 1.18 percentage points on average (Table 2).
- Vision-language PFMs (CONCH, KEEP) encode the least center information at the representation level (low FM-SI, high RI), while KAIKO encodes the most.
- ComBat harmonization does not reliably mitigate center shortcuts: AUCC improves in only 14 of 48 configurations (mean −0.63 pp), with gains concentrated on CAMELYON16 and near-systematic losses on TCGA-BRCA.
Strengths
Genuinely controlled experimental design
Training sets are resampled to target class–center Cramér's V ∈ {0, 0.25, 0.5, 0.75, 1} while the test fold always stays at V = 0 (§2.2), so performance degradation is cleanly attributable to shortcut reliance rather than confounded by test-time distribution shift.
Broad, current benchmark grid
Six frozen PFMs (two vision-language: CONCH, KEEP; four vision-only: VIRCHOW2, H-Optimus-1, KAIKO, UNI-2) × four public datasets × two MIL aggregators × 5 correlation levels × 10 samplings × 5-fold CV (§3.2, Table 2) — 48 AUCC configurations covering state-of-the-art encoders.
Useful new summary metric
Honest negative result on ComBat
Candid limitations section
Weaknesses
No uncertainty quantification anywhere
Fragile TCGA-BRCA cohort
Correlated-sampling algorithm unspecified
ComBat protocol under-documented
Modality conclusion rests on n = 2
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This CASEIB 2026 proceedings paper (arXiv:2609.28231) asks whether acquisition-center information embedded in pathology foundation models (PFMs) leaks into weakly supervised whole-slide-image (WSI) classification. Three claims anchor the work: (1) center-related signal in frozen PFM patch embeddings propagates to slide-level predictions, with balanced accuracy (BACC) degrading as the induced class–center correlation grows (§4.2, Fig. 3); (2) the MIL aggregator is itself a robustness factor — ABMIL beats TransMIL on average AUCC for five of six PFMs, by +1.18 pp on average (Table 2); (3) ComBat harmonization is an unreliable fix, improving AUCC in only 14 of 48 configurations (mean −0.63 pp; §4.3). A secondary observation is that the two vision-language PFMs (CONCH, KEEP) show the least center-dominated representations (§4.1, Fig. 2).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is AUCC in this paper?
AUCC (Area Under the Cramér's V Curve) is the paper's proposed summary metric. Models are trained with increasingly strong spurious class–center correlation (Cramér's V from 0 to 1 in five steps) while the test set always stays uncorrelated; balanced accuracy is measured at each level and the area under the resulting curve is computed by trapezoidal integration. A higher AUCC means both better absolute performance and less degradation when center shortcuts are available during training.
Did ComBat harmonization fix the center bias?
No. Applied to the foundation-model embeddings, ComBat worsened AUCC in 34 of 48 model–aggregator–dataset configurations and improved it in only 14 (average change −0.63 percentage points). Improvements were frequent on CAMELYON16 but nearly systematic losses appeared on TCGA-BRCA, so the authors conclude generic feature harmonization is not a reliable mitigation for center-related shortcuts.
Which pathology foundation models were most robust to center bias?
The two vision-language models, CONCH and KEEP, occupied the most robust region on the representation-space metrics used (low FM-SI, high RI), and CONCH achieved the best overall downstream robustness (average AUCC 85.00 with ABMIL). KAIKO showed the strongest center-related encoding, followed by H-Optimus-1.
What datasets were used in the study?
Four public binary whole-slide-image classification tasks, each restricted to two acquisition centers: AI4SKIN (benign vs malignant skin lesions), CAMELYON16 (breast cancer lymph-node metastasis), TCGA-BRCA (invasive ductal vs lobular carcinoma), and TCGA-NSCLC (lung adenocarcinoma vs squamous cell carcinoma), totaling between 130 and 599 slides per dataset.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 2 and 3 (representation-level FM-SI/RI scatter and BACC-vs-V curves) could not be analyzed — only captions were extracted — so representation-level conclusions rest on the authors' verbal summary.
- Text extraction shows OCR corruption in references and headers (mangled volume/issue fields, placeholder DOIs), though the main body and Table 2 extracted cleanly.
- This is a four-page conference proceedings contribution (CASEIB 2026); an extended version may contain protocol details (sampling algorithm, ComBat settings) absent here.
- No supplementary material was included in the provided text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.