Med-AR: Autoregressive Vision-Language Pretraining for Long-Tailed Chest X-Ray Classification and Uncertainty-Aware Evaluation
Janhavi Prabhu, Sahil, Akshay V, Shivam Shukla, Manoj Tadepalli, Preetham Putha
cs.CV · cs.AI
Quick verdict
Proposes Med-AR autoregressive CXR pretraining beating contrastive baselines on public long-tailed benchmarks; controlled but confounded by richer supervision.
Summary
Researchers at Qure.ai compare autoregressive vision-language pretraining (Med-AR-8B and Med-AR-2B, InternViT-300M encoders with InternVL3-style 8B/2B decoders) against MedCLIP-style contrastive pretraining for long-tailed multi-label chest X-ray classification. Both recipes are pretrained on 1.4 million internal chest X-rays with reports; the autoregressive variant additionally uses 15-section structured report templates, prominence scores, and roughly 100k region annotations. All encoders are fine-tuned with an identical ML-Decoder head on MIMIC-CXR (110 LLM-extracted labels), CheXpert (115), PadChest (29), and the internal set (40). On all three public datasets, a Med-AR variant attains the best mean AUROC and AUPRC in every prevalence group: Med-AR-8B raises MIMIC-CXR tail AUPRC from 0.1033 (Med-CLIP) to 0.1441, and Med-AR-2B leads throughout on PadChest. Med-AR also achieves lower EAURC selective-prediction error on the public datasets, though Med-CLIP retains a large internal EAURC advantage (0.014 vs 0.070). Against seven further encoders (ARK, CheXFound, EVA-Base, RAD-DINO, BiomedCLIP, MedKLIP, BioViL-T), Med-AR remains strongest. Key limitations: the richer autoregressive supervision and decoder scale are confounded with the pretraining objective (no component ablation), the LLM-derived evaluation labels are unvalidated weak supervision, and results come from single training runs.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- On MIMIC-CXR, CheXpert, and PadChest, Med-AR-8B improves mean AUROC and AUPRC over MedCLIP-style contrastive pretraining in every prevalence group, e.g., MIMIC-CXR tail AUPRC rises from 0.1033 to 0.1441.
- Under a common ML-Decoder head, a Med-AR variant is the best of ten compared encoders in every reported prevalence group on each public dataset (Table 6).
- Both Med-AR variants achieve lower EAURC than Med-CLIP on all three public datasets, while Med-CLIP retains the EAURC advantage on internal data, showing discrimination and selective prediction decouple.
- LLM-expanded, report-derived label sets (110 MIMIC-CXR / 115 CheXpert labels, inter-model κ ≈ 0.83-0.87) form a finer-grained long-tail evaluation probe.
- The results support the complete autoregressive pretraining recipe, not the objective alone — the paper explicitly declines causal attribution (§5.6, §5.7).
Strengths
Matched downstream protocol
Both recipes share the InternViT-300M encoder family and a fixed ML-Decoder head (Asymmetric Loss, batch 2304; §3.1-3.3, §4.3), isolating the comparison to pretraining recipes across 1.87M train / 298k test images (Table 1) — a cleaner design than prior cross-paper comparisons the authors criticize.
Unusually honest negative and mixed results
Internal AUPRC and EAURC favor Med-CLIP (Table 3: overall AUPRC 0.2862 vs 0.2842; Table 5: EAURC 0.014 vs 0.070/0.073), CheXpert AR-2B is significantly worse (p ≈ 1.00, §5.2), and the abstract itself scopes claims to the evaluated protocol and flags metric-dependent internal results.
Careful weak-label methodology
Statistically framed evaluation
Broad encoder comparison
Weaknesses
Objective confounded with supervision
Decoder-scale instability
LLM labels as evaluation ground truth
Single-run results
Reproducibility gaps
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This MICCAI-style submission from Qure.ai asks whether radiology-native autoregressive (AR) vision-language pretraining transfers better to long-tailed chest X-ray classification than MedCLIP-style contrastive pretraining, and whether the paradigms differ in uncertainty-aware (selective-prediction) behavior. The authors pretrain Med-AR-2B and Med-AR-8B (InternViT-300M encoder + InternVL3-style decoders) on 1.4M internal CXRs using 15-section structured report templates, abnormality-focused targets, prominence scores (0-5), and ~100k region annotations (§3.3, §4.1). The contrastive baseline re-implements MedCLIP with the same backbone plus CXR-BERT (§3.4). Both are fine-tuned with an identical ML-Decoder head on MIMIC-CXR (110 LLM-derived labels), CheXpert (115), PadChest (29 retained), and the internal set (40). Central claims: (1) a Med-AR variant is best in mean AUROC and AUPRC in every prevalence group on each public dataset (Tables 3/6); (2) MIMIC-CXR tail AUPRC rises 0.1033 → 0.1441; (3) Med-AR attains lower EAURC on all public datasets; (4) internal results are metric-dependent, with Med-CLIP keeping overall/tail AUPRC and EAURC advantages. The abstract is notably careful — claims are scoped to the evaluated protocol and the internal caveat is stated up front.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Med-AR and how does it differ from MedCLIP?
Med-AR is a radiology-specific autoregressive vision-language model: an InternViT-300M vision encoder paired with a 2B or 8B language decoder, pretrained on 1.4 million chest X-rays using structured report templates, abnormality lists, prominence scores, and bounding-box annotations. The paper's Med-CLIP baseline is a MedCLIP-style contrastive model with the same vision backbone trained on the same images by aligning image and report embeddings. In both cases only the vision encoder is kept for downstream classification, attached to the same ML-Decoder head.
How much does Med-AR improve rare-finding (tail) detection on chest X-rays?
On the paper's LLM-expanded MIMIC-CXR benchmark (110 report-derived labels), Med-AR-8B raises mean tail-label AUPRC from 0.1033 (Med-CLIP baseline) to 0.1441 and tail AUROC from 0.8733 to 0.8967; on CheXpert, tail AUPRC improves from 0.0922 to 0.1151 with the 8B model. Note these gains are measured against LLM-extracted report labels rather than expert image annotations.
Does autoregressive pretraining always beat contrastive pretraining for chest X-ray classification?
Not always. On the three public benchmarks (MIMIC-CXR, CheXpert, PadChest) a Med-AR variant wins every reported prevalence group, but on the authors' internal 1.4M-image dataset results are mixed: Med-AR-2B has slightly better AUROC while Med-CLIP keeps better overall and tail AUPRC and a much better selective-prediction score (EAURC 0.014 vs 0.070). The smaller 2B variant also loses to Med-CLIP on CheXpert discrimination.
Is code or data available for Med-AR?
The paper does not mention a code release. The authors state they plan to release the derived MIMIC-CXR and CheXpert label annotations and split identifiers after publication, subject to the source datasets' licensing terms. The 1.4M-image pretraining corpus is internal to Qure.ai and proprietary, so full retraining by external groups is not possible.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Mid-paper sections (§3.5-3.7 tail-grouping details, parts of §5.3-5.4 including the DeLong results narrative) and portions of the supplementary per-label tables were truncated in extraction.
- Figures (label distributions, concordance plots, EAURC curves) could not be analyzed; assessments rest on tables and prose.
- Table extraction interleaves some columns; numbers cited were cross-checked against the prose where both were available.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.