Integrating knowledge from case reports: a medical ontology based multimodal information system with structured summary
Shuyu Guo, Lan Huang, Yichen Liu, Hanbin Ma, Tian Bai
cs.AI
Quick verdict
CRFinder: 52,949-case-report multimodal database with ontology-linked images/entities; valuable resource, but non-standard evaluation, no code/URL released.
Summary
The authors present CRFinder, a database and web platform that restructures 52,949 open-access PMC case reports (2000–2021) into structured multimodal summaries for clinician retrieval, targeting rare-disease diagnosis support. The pipeline extracts key sentences (captions, figure mentions, case descriptions, main findings) via rules plus MeSH co-occurrence scoring; tags disease, symptom and body-system entities with QuickUMLS normalized to UMLS CUIs, plus biological entities via PubTator; and classifies 58,601 medical images across five modalities (30,200 CT, 17,712 microscopy, 12,136 MRI, 4,357 ultrasound, 2,595 endoscopy) with CNNs trained on keyword-derived labels, EfficientNet-b0 reaching 0.859 accuracy on 1,966 test figures. The database covers 55,212 patient cases and 935 GARD-listed rare-disease types. Evaluation reports 0.632 main-finding extraction accuracy (261 overlapping reports, cosine threshold 0.5), average retrieval precision 0.816 and recall 0.281 over 100 queries using bespoke relevance definitions, and a ten-participant usage study. Main limitations: evaluation metrics deviate from standard IR definitions; image-classifier labels come from the same keyword heuristic used for training, so no human-verified gold standard exists; basic-information and entity-linking accuracy go unquantified; no code, dataset splits, or live system URL is provided; coverage ends in 2021 while the paper appeared in 2026.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 4
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- CRFinder is the first comprehensive case-report database, integrating structured multimodal summaries (medical images, clinical and biological entities, key sentences) from 52,949 open-access PMC case reports published 2000–2021.
- Organizing summaries under disease, symptom and body-system ontologies enables retrieval when clinicians cannot name the disease, supporting rare-disease identification, with 935 GARD-listed rare-disease types covered.
- EfficientNet-b0 classifies figures into five imaging modalities with 0.859 accuracy, the best of seven CNN backbones evaluated on 1,966 test figures.
- Keyword-based retrieval achieves average precision 0.816 and recall 0.281 over 100 clinical queries, and a ten-participant study indicates the system supports both doctors' and students' case-report retrieval.
Strengths
Large-scale, clinically targeted curation
52,949 PMC open-access case reports (2000–2021) become 55,212 patient cases with 58,601 modality-classified images (30,200 CT, 17,712 microscopy, 12,136 MRI, 4,357 ultrasound, 2,595 endoscopy; Section 3.1) and 935 GARD rare-disease types — a substantial resource aimed at a genuine rare-disease retrieval gap.
Ontology-guided browsing addresses the keyword-search failure mode
Hierarchical disease/symptom/body-system filters (CTD, Symptom Ontology, Human Developmental Anatomy Ontology via BioPortal; UMLS CUI normalization) let users retrieve without knowing a disease name, demonstrated by the COVID-19 ground-glass-opacity walkthrough (Section 3.2, Figure 5); Table 7 shows Open-i and YIF lack this guidance.
Systematic backbone comparison with candid reporting of weak numbers
Multi-level evaluation attempt
Transparent scope discussion
Weaknesses
Training/test label circularity in the image classifier
Non-standard evaluation metrics
Unevaluated core pipeline stages
No released artifacts
Overclaimed novelty, feature-only comparison
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
CRFinder (Sections 1–2) is presented as 'the first comprehensive database focusing on case reports': 52,949 open-access PMC case reports (2000–2021, English) distilled into structured multimodal summaries — 58,601 medical images in five modalities, UMLS-mapped disease/symptom/body-system entities, PubTator biological entities, and up to three MeSH-filtered main-finding sentences per report — organized under CTD and BioPortal ontologies and served through a browse-and-search web interface (Section 3.2). The stated audience is junior clinicians confronting rare diseases. Central claims: (1) a novel case-report-focused multimodal resource; (2) ontology-guided browsing enables retrieval when the disease name is unknown (COVID-19 ground-glass-opacity walkthrough, Figure 5); (3) EfficientNet-b0 best classifies imaging modality at 0.859 accuracy; (4) keyword retrieval attains average precision 0.816 and recall 0.281 over 100 clinical queries, with a positive ten-participant usage study.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is CRFinder?
CRFinder is a database and web search platform described in this paper. It converts 52,949 open-access case reports from PubMed Central (published 2000–2021) into structured multimodal summaries: 58,601 medical images sorted into five imaging types, disease/symptom/body-system entities mapped to medical ontologies, biological entities, and key conclusion sentences. It is intended to help clinicians, especially juniors, find similar cases when diagnosing rare diseases.
How much data does the CRFinder database contain?
Per the paper: 52,949 case reports covering 55,212 patient cases; 58,601 medical images (30,200 CT, 17,712 microscopy, 12,136 MRI, 4,357 ultrasound, 2,595 endoscopy; some compound figures carry multiple labels); 5,538 disease, 3,101 symptom and 4,929 body-system entities; and 935 rare-disease types matching the US GARD catalog.
Is CRFinder better than PubMed or Open-i for finding case reports?
The paper's comparison (its Table 7) is a feature checklist, not a head-to-head test. Unlike PubMed and SIBiLS, CRFinder supports image browsing plus ontology-guided filtering; versus Open-i it adds ontology-based browsing and case-report-focused structured summaries, but Open-i offers 8 imaging filters against CRFinder's 5 and broader collections. No user study comparing the tools was run, so no performance superiority is demonstrated.
Is the code or data for this paper publicly available?
The source case reports are public (the PMC open-access FTP subset) and the pipeline uses existing tools (QuickUMLS, PubTator, standard CNN architectures), but the paper contains no released code, no dataset splits, and no URL to the CRFinder system itself, so the resource cannot currently be accessed or reproduced as published.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–7 (ontology distributions, confusion matrix, interface screenshots) are images and could not be verified from the extracted text.
- Table 5 lists only 10 of the 100 evaluation queries; the full query set and per-query judgments are unavailable, limiting assessment of the headline retrieval numbers.
- No URL or access point for CRFinder appears in the text; public availability of the system could not be confirmed.
- The five per-modality figure counts sum to 67,000, exceeding the stated 58,601 medical images; the paper attributes this to compound multi-label figures, which is plausible but unverifiable from the text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.