LLM4CKD: Large Language Models for Early Stage Chronic Kidney Disease Screening
Muhammad Ashad Kabir, Sirajam Munira
cs.AI · cs.LG
Quick verdict
Claims LLMs match ML/DL for low-data CKD screening; rigorous multi-baseline benchmark, but small cohorts and model-dependent gains temper the headline.
Summary
The paper introduces LLM4CKD, an evaluation framework applying instruction-tuned LLMs to early-stage CKD screening from structured tabular features under zero-shot and few-shot in-context learning, benchmarked against eight ML models, four tabular DL/foundation models (TabNet, NODE, TabPFN, SAINT), and five published rule-based screening tools. Using a 284-participant Bangladeshi community cohort (112 early-stage CKD cases; 24 features, eGFR/uACR excluded to prevent label leakage) and the UCI CKD dataset (400 records) for independent evaluation, the authors test five LLMs (Gemma-2-9B, Llama-3-8B, Qwen-3-8B, Mistral-7B, GPT-4o-mini) across prompt styles, serialization templates, and all/selected feature sets, with five shared seeds, paired permutation tests, and mixed-effects analysis. Key results: selected-feature prompting improves most LLMs (Llama-3 Chat+Text 0.51→0.74 balanced accuracy); Qwen-3 four-shot reaches 0.814 balanced accuracy with Brier 0.154; TabPFN overtakes the LLMs at 32 training samples (0.820 balanced accuracy, Brier 0.134 per Table IV); several zero-shot LLMs (Qwen-3, Mistral, GPT-4o-mini, Gemma-2) show significantly lower Brier loss than all five screening tools, though the best rule-based tool still tops them on balanced accuracy. Limitations: small primary cohort (~57-sample test set), a hospital-based external dataset without stage information, model- and prompt-dependent stability, under-specified LLM feature-ranking derivation, and no missing-data robustness or prospective evaluation. Code is public.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- LLMs under zero/few-shot in-context learning achieve competitive early-stage CKD screening with a handful of examples, often matching or outperforming ML, DL, and tabular foundation models in the lowest-data settings (Fig. 3, Table IV).
- Clinically selected compact features improve or maintain zero-shot LLM performance for most models, but the effect depends jointly on model, prompt style, and serialization template (Fig. 2).
- The best zero-shot LLMs achieve significantly lower Brier loss than five established rule-based CKD screening tools on the full primary cohort, though the best tool still leads on balanced accuracy (Table VI).
- LLMs prioritize clinically meaningful CKD risk factors but weight them differently from the supervised ML reference ranking (Table V, Fig. 5).
- LLM performance does not consistently improve with more in-context examples, whereas supervised tabular models scale steadily with labeled data (Fig. 3, Fig. 6b).
Strengths
Unusually broad, paired benchmark design
Five LLMs x 2 prompt styles x 2 serialization templates x all/selected features vs 8 ML, 4 DL/TFM baselines, 5 rule-based tools, plus an independent dataset — with shared 80/20 splits and identical training/in-context samples per seed across all models (Section II-F), enabling genuinely paired comparison.
Statistical reporting above genre norm
Paired permutation tests on per-sample Brier loss, mixed-effects models with 95% CIs (Fig. 4), BCa CIs and standard deviations in Table IV, and calibration (Brier, ECE) reported alongside balanced accuracy over five pre-specified seeds.
Label-leakage discipline
Clinical plausibility analysis beyond accuracy
Calibrated, non-triumphal framing
Weaknesses
Small primary cohort and test set
Weak external validation choice
Under-specified LLM feature-ranking method
No multiple-comparison control
Internal numeric inconsistency
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
LLM4CKD (accepted at IEEE ICDM 2026) is an applied benchmark study asking whether instruction-tuned LLMs can screen for early-stage chronic kidney disease (CKD) from structured tabular features without task-specific training. The authors evaluate five LLMs (Gemma-2-9B, Llama-3-8B, Qwen-3-8B, Mistral-7B, GPT-4o-mini) under zero-shot and few-shot (4/8/16/32 examples) in-context learning, benchmarking against eight scikit-learn ML models, four tabular DL/foundation models (TabNet, NODE, TabPFN, SAINT), five published rule-based screening tools (SCORED, Kshirsagar, Thakkinstian, Kwon, Kearns), and an independent dataset. The central claims are calibrated rather than triumphal: LLMs are competitive in extreme low-data regimes, gains are model- and prompt-dependent, and supervised tabular models win once ~32 labeled samples exist.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Can large language models screen for chronic kidney disease without training data?
According to this study, partially yes: five instruction-tuned LLMs evaluated zero-shot on structured clinical features reached balanced accuracy between roughly 0.73 and 0.79 on a 284-person Bangladeshi cohort, and four of them (including Qwen-3 and Mistral) achieved significantly lower Brier loss than five published rule-based CKD screening tools. However, the best rule-based tool still scored higher on balanced accuracy (0.8103 vs 0.7925), results varied widely by model and prompt format, and a small tabular foundation model (TabPFN) overtook the LLMs once 32 labeled training examples were available.
Which LLM performed best for CKD screening in LLM4CKD?
Qwen-3-8B and Mistral-7B were the strongest overall. Qwen-3 reached 0.814 balanced accuracy with only four in-context examples (AUROC 0.867, Brier 0.154) and was competitive with the best ML and tabular baselines in the lowest-data settings. Mistral attained the highest zero-shot LLM balanced accuracy (0.7925). Performance was model-dependent: Llama-3 and Gemma-2 were less stable across prompt configurations, and extra in-context examples did not reliably help.
What datasets were used in the study and how big are they?
The primary dataset is a community cohort of 284 adults from Mirzapur, Bangladesh — 112 early-stage CKD cases (stages 1-3) and 172 non-CKD individuals, described by 24 clinical and socio-demographic features; direct diagnostic markers (eGFR, uACR) were excluded from model inputs to prevent label leakage. The independent dataset is the UCI CKD dataset with 400 hospital records from India (250 CKD, 150 non-CKD), used only for cross-dataset checks because CKD stage is not reported there.
Do LLMs beat traditional machine learning for low-data tabular prediction?
In this paper's setting, only in the extreme low-data regime: with zero to four labeled examples, LLMs matched or exceeded Random Forest, XGBoost, MLP, TabNet, NODE, SAINT, and TabPFN. As training data grew to 32 samples, supervised models — especially TabPFN (0.820 balanced accuracy) and NODE (0.797) — overtook the LLMs, whose performance did not consistently improve with more in-context examples. The authors frame this as a trade-off between data efficiency (LLMs) and stability/scalability (supervised tabular models).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure panels (Figs. 2-6) were extracted as run-on numeric text; per-bar values quoted in this review were cross-checked against the prose, but some individual figure-level numbers may be misaligned by extraction.
- Table I marks selected features by underline, which does not survive PDF extraction, so the exact selected-feature list could not be independently verified from the provided text (only its provenance from prior work [9] is described).
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.