Syndrome, Synergy, and Safety: Structured Reasoning and Knowledge-Driven Alignment for TCM Prescription Generation
Zheng Chen, ZhiCheng Du, Haoxuan Li, Haoxuan Li, Peiwu Qin
cs.CL · cs.AI
Quick verdict
四级中药处方流水线显示出实际效果，但“7B超越GPT-5”的头条新闻依赖于不一致的PQS报告和极其微小的优势。
Summary
该论文针对基于LLM的中医处方生成中的三个缺口：缺乏符合理法方药（li-fa-fang-yao）范式的可审计推理、缺乏纵向随访调整（随证加减）、以及未执行的禁忌规则（十八反/十九畏）。作者使用QLoRA对12个开源模型进行了四个递进阶段的微调：在4万条临床记录上进行基准SFT；从DeepSeek-R1提取范式引导的、经人工验证的思维链（chain-of-thought）；在伪ID患者轨迹上进行动态SFT；以及通过注入禁忌草药并利用基于规则的检查器进行评分，由此产生的偏好对进行DPO对齐。在871个样本的预留测试集和53个对抗性探测中，处方质量（PQS）、推理对齐（CQS）和违规率（VR）在各阶段均有提升；核心结果是经过微调的Mistral-7B在所有三个指标上超越了零样本（zero-shot）GPT-5（例如，PQS为0.701对0.664）。主要局限性（作者已在很大程度上承认）：顺序流水线混淆了各阶段的贡献；测试集仅包含初诊数据，因此纵向主张未经过直接评估；自定义指标部分偏向于结构化输出；核心比较的PQS报告在不同表格间存在内部不一致；临床数据保持私有。
Review complete
Assessment at a glance
6
risk flags identified
● high 1
● medium 4
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A progressive four-stage pipeline (SFT, PG-CoT, Dynamic SFT, K-RL) produces auditable li-fa-fang-yao reasoning, longitudinal adjustment capability, and reduced contraindication violations across 12 fine-tuned models.
- Fine-tuned Mistral-7B at the K-RL stage surpasses zero-shot GPT-5 on all three evaluation metrics (PQS 0.701 vs 0.664, CQS 0.766 vs 0.671, VR 0.271 vs 0.285; Table 4).
- Encoding Shi Ba Fan / Shi Jiu Wei rules as synthetic DPO preference pairs reduces adversarial violation rates on high-vulnerability models (e.g., Mistral-7B VR 0.432 to 0.271), though effects are model-dependent.
- The li-fa-fang-yao constraint is a performance-enhancing inductive bias: paradigm-guided CoT beats free-form CoT by +0.29 auditability, -0.12 logical inconsistency, and +0.23 PQS on Mistral-7B (Appendix D).
Strengths
Clinically literate problem decomposition with a supporting pilot
The three gaps map to concrete TCM concepts (li-fa-fang-yao, sui zheng jia jian, Shi Ba Fan) and each gets a targeted mechanism. Appendix D (Table 7) quantifies the paradigm constraint on Mistral-7B: auditability +0.29, logical inconsistency -0.12, PQS 0.423 to 0.651 versus free-form CoT.
Unusually broad and complete evaluation grid
12 fine-tuned models across 6 families and 3 scale tiers, stage-wise ablation (Tables 2, 3, 5, 9), a 53-probe adversarial set with validated 100% trigger rate, and a structured expert study (Appendix K, Cohen's kappa = 0.77).
Candid limitations section
Rule-based DPO signal is a sensible, cheap safety idea
Weaknesses
Sequential pipeline prevents stage attribution
Metric circularity favors the proposed method
Longitudinal claim is not directly evaluated
Safety checker coverage is partial and saturates
No uncertainty quantification
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper targets LLM-based Traditional Chinese Medicine (TCM) prescription generation and claims three clinically grounded gaps: no auditable reasoning along the li-fa-fang-yao paradigm (SR), no follow-up adjustment via sui zheng jia jian (LA), and unenforced contraindication rules Shi Ba Fan / Shi Jiu Wei (SC). The remedy is a progressive QLoRA pipeline - baseline SFT, then PG-CoT (paradigm-constrained CoT distilled from DeepSeek-R1 with human verification), then Dynamic SFT on pseudo-ID patient trajectories, then K-RL (DPO on synthetic safety pairs) - evaluated on 12 fine-tuned models and 6 zero-shot baselines over an 871-case test set and 53 adversarial probes. The headline claim (Abstract, Section 5.2, Table 4) is that fine-tuned Mistral-7B surpasses zero-shot GPT-5 on all three metrics: PQS 0.701 vs 0.664, CQS 0.766 vs 0.671, VR 0.271 vs 0.285.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Does a 7B model really beat GPT-5 at TCM prescriptions in this paper?
Per its Table 4, the fine-tuned Mistral-7B scores higher than zero-shot GPT-5 on the paper's three custom metrics (PQS 0.701 vs 0.664, CQS 0.766 vs 0.671, VR 0.271 vs 0.285). Caution is warranted: the comparison pits in-domain fine-tuning against zero-shot use, the CQS metric structurally favors models trained to emit the required structured format, the VR margin is about one probe out of 53, and the reported PQS for the fine-tuned model is inconsistent across the paper's own tables (0.701 in Table 4 vs 0.350 in Table 9 for the K-RL stage).
What is Shi Ba Fan and how is it used in this work?
Shi Ba Fan (Eighteen Incompatibilities) is a classical set of herb pairs in Traditional Chinese Medicine considered absolutely contraindicated, e.g., licorice with Gan Sui, Da Ji, Yuan Hua, or Hai Zao, and aconite with Bei Mu, Gua Lou, Ban Xia, or Bai Ji. The paper encodes these plus the Shi Jiu Wei pairs and a toxic-herb list as a rule-based scorer (TCMSafetyChecker), used both to grade outputs and to build DPO preference pairs by injecting violating herbs into otherwise valid prescriptions.
Is the code or data available for this paper?
Not at review time. The authors promise, upon publication, to release training code, evaluation scripts, model adapters, the TCMSafetyChecker rule set, the 53-prompt adversarial probe set, and an anonymized subset of the clinical data. The full 124,593-record clinical dataset stays private for privacy reasons, so full replication of the headline tables is not currently possible.
What are the paper's main limitations?
The authors acknowledge: stage contributions cannot be separated because each training stage builds on the previous one (and the headline gain mostly comes from plain domain SFT); the test set contains only initial-visit cases, so follow-up adjustment ability is not directly tested; the safety gains are model-dependent and violation rates remain high for many models; metrics measure similarity to model-distilled reference reasoning rather than clinical correctness; and the clinical data comes from specific outpatient departments, so generalization is unverified.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Table 2 as extracted is garbled (PQS and CQS sub-columns interleaved); values were cross-validated against Tables 9 and 12 and Appendix F prose.
- Figures 1-5 are not analyzable from the extracted text, including the stage-wise CQS trajectories in Figure 4.
- Some appendix tables may be truncated in extraction; internal inconsistencies reported here were verified against the paper's own prose wherever possible.
- Model names (Qwen3.5, GPT-5, LLaMA-4-Scout) and the arXiv ID reflect the source text as given; no external verification was performed.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.