LingLan: An Advancing Traditional Chinese Medicine Diagnosis LLM with Multimodal Data
Zheng Chen, Zhicheng Du, Haoxuan Li, Yingshan Liang, Peiwu Qin
cs.AI
Quick verdict
Claims 103.5% relative accuracy gain for a TCM diagnosis LLM (62.72%); promising, but malformed metrics and inconsistent confusion-matrix counts need scrutiny.
Summary
The paper presents LingLan-14B, a Traditional Chinese Medicine (TCM) diagnosis LLM built by LoRA fine-tuning Qwen3-14B on structured clinical records. The authors first propose UFMD, a pipeline that uses DeepSeek-OCR to convert tongue and pulse images plus paper/electronic records into standardized textual findings, filters hallucinations via Sentence-BERT cosine-similarity thresholding, and applies manual expert verification, reportedly structuring over 120,000 TCM records. Records are templated to combine inspection, palpation, and inquiry information following the I-AOI-P diagnostic workflow. On a 20% stratified test split labeled by GB/T 15657-2021 (~24,000 records), LingLan-14B reports 62.72% accuracy, 82.03% recall, and 0.8202 F1, versus 30.82% accuracy for un-finetuned Qwen3-14B (a 103.5% relative gain) and 32.66% accuracy for the TCM-specific BianCang-14B-Instruct; general LLMs up to DeepSeek-R1-671B stay near 27-33%. An ablation shows structured data roughly doubles diagnostic accuracy (0.3790 to 0.6334). Main limitations: the proposed UDI/PCDI metric formulas are mathematically malformed; the cited confusion-matrix diagonal counts (e.g., 2,850 correct BN) contradict the test-set category totals and the 82% recall; no significance tests, patient-level split safeguards, or majority-class baseline despite a 48%-dominant class; dataset and weights are not released. Results are promising but the reported numbers require verification.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- UFMD converts tongue/pulse images and mixed-format TCM records into expert-verified structured data, processing over 120,000 clinical records (Sec. III-A).
- LoRA SFT of Qwen3-14B on I-AOI-P-structured records raises diagnostic accuracy from 30.82% to 62.72%, a 103.5% relative improvement (Table III).
- LingLan-14B outperforms the TCM-specific BianCang-14B-Instruct and general LLMs up to DeepSeek-R1-671B on all five reported metrics (Table III).
- The structured I-AOI-P data format, not fine-tuning alone, drives the gain: unstructured SFT reaches only 0.3790 diagnostic accuracy versus 0.6334 (Table IV).
Strengths
Practical data-engineering contribution at scale
UFMD (Sec. III-A) couples DeepSeek-OCR extraction with SBERT cosine-similarity hallucination gating and expert proofreading, reportedly structuring over 120,000 real TCM records — directly targeting the manual-curation bottleneck the paper itself identifies in Sec. II.
Ablation directly supports the core hypothesis
Table IV holds Qwen3-14B and hyperparameters constant: structured I-AOI-P data lifts diagnostic accuracy from 0.3790 to 0.6334 and syndrome-differentiation accuracy from 0.2460 to 0.6210, isolating data structure rather than SFT per se as the driver.
Tiered and relevant baseline set
Standardized, auditable label space design
Extreme training efficiency
Weaknesses
Novel metrics are mathematically malformed
Internal inconsistency in headline results
No statistical rigor or trivial baseline
Leakage and evaluation protocol unaddressed
Reproducibility blockers and framing overstatement
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
LingLan (arXiv 2609.25715, cs.AI) targets AI-assisted Traditional Chinese Medicine (TCM) diagnosis. Two contributions: (1) UFMD, a data-unification pipeline that converts tongue/pulse images and paper or electronic records into standardized structured text via DeepSeek-OCR, with Sentence-BERT-based hallucination gating (cosine similarity above an unspecified threshold) plus expert proofreading — reportedly structuring over 120,000 TCM records (Sec. III-A); (2) LingLan-14B, a LoRA fine-tune of Qwen3-14B trained on records templated to follow the I-AOI-P (Inspection, Auscultation/Olfaction, Inquiry, Palpation) workflow. Central claims: 62.72% accuracy versus 30.82% for un-finetuned Qwen3-14B (the advertised 103.5% relative gain), F1 0.8202, and superiority over BianCang-14B-Instruct and general LLMs up to DeepSeek-R1-671B (Table III), plus an ablation attributing the gain to structured I-AOI-P data (Table IV). Note that despite the multimodal branding, the diagnosis model is text-only; images are processed upstream by DeepSeek-OCR into text, so the multimodality lives in the data pipeline, not the model.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is LingLan-14B?
LingLan-14B is a Traditional Chinese Medicine (TCM) diagnosis language model created by LoRA fine-tuning Qwen3-14B on over 120,000 structured TCM clinical records. Each record combines tongue findings, pulse findings, and inquiry answers following the classical I-AOI-P (inspection, auscultation/olfaction, inquiry, palpation) workflow; the paper reports 62.72% diagnostic accuracy versus 30.82% for the un-finetuned base model.
How much better is LingLan-14B than other models?
On its ~24,000-record stratified test set, the paper reports LingLan-14B at 62.72% accuracy and 0.8202 F1, versus 32.66% accuracy for the TCM-specific BianCang-14B-Instruct and roughly 27-33% accuracy for general LLMs including DeepSeek-R1-671B. Independent verification is warranted: the paper's custom metric formulas contain mathematical errors and its confusion-matrix counts appear inconsistent with its own recall figure.
Is the LingLan dataset or model publicly available?
No. The authors state the clinical dataset cannot be released due to patient privacy, though an anonymized version is planned after a formal privacy assessment; LingLan-14B weights are promised after paper acceptance.
What is UFMD in this paper?
UFMD (Unification Framework for Multimodal Data) is the authors' data pipeline: DeepSeek-OCR converts tongue and pulse images plus paper/electronic medical records into standardized structured text, extractions are screened with a Sentence-BERT cosine-similarity threshold to catch hallucinations, and experts manually verify the output — producing a unified digital record of the TCM diagnostic process.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-4 (diagnosis motivation, UFMD overview, test distribution, confusion matrices) are not analyzable from the extracted text; the Fig. 4 inconsistency is judged from prose numbers only.
- The metric equations may be distorted by text extraction, but as printed they are mathematically invalid and cannot support the reported UDI/PCDI values.
- References and related-work claims were checked only against the provided text; no external verification was performed.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.