A visual large language foundational model for medical image recognition using clinician-contributed online resources
Lingxuan Hou, Yuhua Xie, Yue Hu, Yan Zhuang, Junqi Li, Chengzhi Xia, Binh Phu Nguyen, Abubakar Siddique, et al.
cs.AI
Quick verdict
Claims SOTA medical VLM from a 1M-pair LLM-generated dataset; data/code access is suspended over permissions and key evals risk circularity.
Summary
The paper introduces ThoughtMed-1M, a medical visual question answering dataset of 1,018,472 VQA pairs built from 68,946 clinician-authored cases (64,493 Radiopaedia cases; 4,453 figure1.com posts) spanning 227,229 images. VQA pairs were generated primarily by ChatGPT 5.2 — selected over Claude Sonnet 4.5, Qwen3-VL-Max, and DeepSeek-V4 via ratings from ten radiologists — then refined by MedGemma-27B-VL and retained only under unanimous approval of three LLM judges. Answers embed bounding boxes and stepwise reasoning. The authors fine-tune Gemma4-31B-PT through domain-adaptive pretraining (Medical Domain Corpus, MIMIC-III), two-stage SFT, and GRPO-based reinforcement fine-tuning to create FOLTMed. On OmniMedVQA's 42 datasets it reports the best macro accuracy (0.854) versus MedGemma-27B (0.807) and ChatGPT-5.2 (0.824), ranking first on 17 datasets, with anatomy-identification accuracy of 84.93%. On a held-out set of 120 Radiopaedia cases it leads BERTScore/FActScore/AlignScore by roughly 0.03–0.05, and clinician questionnaires rated its responses favorably. Main limitations, largely acknowledged: data, code, and weights are all on hold because permissions from Radiopaedia and figure1.com remain unresolved — making the work currently unverifiable and contradicting the 'largest publicly available' claim; benchmark comparisons lack significance testing; and the open-ended evaluation uses references generated by the same LLM pipeline that built the training data, with ChatGPT-5.2 acting as both baseline and judge.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- ThoughtMed-1M is a 1,018,472-pair long-form medical VQA dataset with embedded clinical reasoning and bounding-box grounding, built from 68,946 clinician-authored Radiopaedia and figure1.com cases via an LLM-generation plus multi-round QC pipeline.
- FOLTMed (Gemma4-31B-PT + DAPT + two-stage SFT + GRPO) achieves state-of-the-art macro accuracy of 0.854 across 42 OmniMedVQA datasets, exceeding MedGemma-27B (0.807) and ChatGPT-5.2 (0.824).
- On a 120-case held-out Radiopaedia test set, FOLTMed outperforms state-of-the-art models by approximately 0.03–0.05 on BERTScore, FActScore, and AlignScore, and receives favorable clinician questionnaire ratings.
- The end-to-end pipeline constitutes a scalable, generalizable paradigm for converting heterogeneous clinician-authored online cases into multimodal training data.
- ThoughtMed-1M is claimed as the largest publicly available medical VQA dataset — a claim the paper's own Data Availability section currently suspends.
Strengths
Genuine scale and breadth of the curated resource
1,018,472 VQA pairs from 68,946 clinician-authored cases (64,493 Radiopaedia + 4,453 figure1.com; 227,229 images), covering full anatomy (Fig 1b) and broad modality spectrum (Fig 1c), with substantially longer answers than PMC-VQA/SLAKE/VQA-RAD (Fig 1d) and embedded bounding boxes.
Clinician-driven generator selection rather than vibes
Ten radiologists from Guang'anmen Hospital rated VQA pairs from 4 LLMs across 8 items / 4 dimensions (Fig 2a-c): ChatGPT 5.2 mean 8.05 (95% CI 7.95–8.15) vs Claude Sonnet 4.5 at 7.64, Qwen3-VL-Max 6.87, DeepSeek-V4 6.41 — a documented basis for choosing the generator.
Comprehensive head-to-head benchmarking including proprietary models
Concrete, inspectable reward design for the RFT stage
Unusually candid limitations and disclosure sections
Weaknesses
Nothing is verifiable: dataset, code, and weights all withheld
Circular open-ended evaluation favors the authors' model
No statistical testing behind the SOTA claim
Scale-stage QC is entirely LLM-based and self-referential
Provenance choices that reduce information content
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper contributes two artifacts: **ThoughtMed-1M**, a long-form medical VQA dataset of 1,018,472 pairs built from 68,946 clinician-authored cases (64,493 Radiopaedia cases with 219,558 images; 4,453 figure1.com posts with 7,671 images; 227,229 images total), and **FOLTMed**, a multimodal model produced by DAPT (Medical Domain Corpus → MIMIC-III), two-stage SFT, and GRPO reinforcement fine-tuning of Gemma4-31B-PT. Central claims: (1) ThoughtMed-1M is the largest medical VQA resource, uniquely embedding bounding boxes and stepwise reasoning; (2) FOLTMed is state-of-the-art on 42 OmniMedVQA datasets (macro accuracy 0.854); (3) it leads open-ended metrics by 3–5% on a held-out test set; (4) the curation pipeline is a scalable paradigm.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is ThoughtMed-1M?
ThoughtMed-1M is a medical visual-question-answering dataset of 1,018,472 question-answer pairs, generated mainly with ChatGPT 5.2 from roughly 69,000 clinician-authored cases on Radiopaedia and figure1.com. Answers include bounding-box annotations and stepwise clinical reasoning. Important: as of this preprint, its release is entirely on hold while the authors seek data-use permissions from the two source platforms.
Is FOLTMed's code or the model weights available?
No. The Code Availability section states that source code, trained weights, and repositories remain private, with all sharing suspended pending resolution of data-use and permission issues with Radiopaedia and figure1.com — access is currently denied even to editors and reviewers.
How accurate is FOLTMed compared with other models?
The authors report the best macro accuracy across the 42 OmniMedVQA benchmark datasets (85.4%), ahead of ChatGPT-5.2 (82.4%) and MedGemma-27B (80.7%), plus 3–5% higher scores on factuality and similarity metrics for open-ended answers on their own 120-case test set. These figures come from the paper alone and cannot currently be independently verified because no artifacts are released.
Can FOLTMed be used for real clinical diagnosis?
No. The authors explicitly state that task-level accuracies of roughly 80–90% are insufficient for use as an independent clinical diagnostic standard, and position FOLTMed strictly as a research foundation model pending blinded reader studies and prospective clinical evaluation.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Supplementary Tables 1–8, Note 1, and Supplementary Fig. 1 are referenced but not included in the provided text, so per-dataset accuracies (Supplementary Table 5), full prompts, and training hyperparameters could not be checked.
- Figures appear in the extraction only as captions and schematic text; the visual content of Figs 1–4 and Extended Data Figs 1–5 could not be independently verified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.