MedAgent-R1: Faithfulness-Aware Reinforcement Learning for Evidence-Grounded Medical Reasoning
Jiangwang Chen, Chenghao Zhang, Hengxing Cai
cs.AI
Quick verdict
Proposes faithfulness-gated RL that cuts citation fabrication 31.8%→4.7% at unchanged accuracy; well-controlled and mostly convincing, but LLM-judge-dependent.
Summary
The paper identifies 'confident hallucination' in RL-trained medical retrieval agents: outcome-only rewards lift average accuracy from 69.64% to 74.56% across four medical QA benchmarks (Table 1) while citation fabrication doubles from 16.5% to 31.8% and evidence completeness (EC-F1) falls from 65.1 to 58.7 (Table 2), as the Qwen2.5-7B policy learns to answer from parametric memory and backfill citations. The remedy is a two-stage pipeline: faithfulness-filtered dual-teacher SFT with observation masking, then GRPO with a gated reward in which accuracy credit requires NLI-verified, explicitly cited evidence, plus retrieval-validity and conciseness terms. On MedFaith-Eval (800 physician-annotated questions, κ=0.72), the full system cuts fabrication to 4.7%, raises EC-F1 to 82.6, holds 75.12% accuracy, and gains 13.2 points on a 1,000-consultation HealthBench safety subset; against GPT-4o with the same agentic loop it wins only Factual Support and Overclaiming, not overall. An inference-time evidence ablation (−16.62 vs −5.18 points) causally supports the mechanism, and a 14B replication shows the pattern persists. Main limitations: all faithfulness endpoints are scored by a proprietary GPT-5.5 judge (partially mitigated by cross-judge and human checks, κ=0.82), backbones are Qwen-only, safety uses a custom internal protocol, and the +0.56 accuracy gain lacks a dedicated significance test.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Outcome-only RL on medical retrieval agents causes confident hallucination: accuracy rises 69.64%→74.56% while citation fabrication rises 16.5%→31.8% and EC-F1 falls 65.1→58.7 on the same Qwen2.5-7B backbone.
- A faithfulness-gated reward (accuracy credit conditioned on NLI-verified, explicitly cited evidence) reduces fabrication to 4.7% and lifts EC-F1 to 82.6 at 75.12% accuracy.
- Faithfulness training transfers to clinical safety: +13.2 points on a 1,000-consultation HealthBench safety subset vs outcome-only (p<0.001, CI [10.5, 15.9]).
- The parametric-shortcut mechanism is causal: inference-time evidence replacement costs the full model 16.62 accuracy points vs 5.18 for outcome-only (Appendix H).
- Under the same agentic retrieval loop, MedAgent-R1 outscores GPT-4o only on faithfulness-specific HRS dimensions (Factual Support 4.55 vs 4.25; Overclaiming 4.40 vs 4.15), not on overall accuracy.
Strengths
Controlled causal isolation of reward design
SFT / outcome-only / full configurations share backbone, data, and retrieval stack (Tables 1-2), cleanly separating reward incentives from domain transfer; the Search-R1 fabrication signal (25.2%) is explicitly flagged as transfer-confounded.
Inference-time evidence ablation is decisive
Appendix H (Table 11): replacing retrieved observations costs the full model 16.62 accuracy points vs 5.18 for outcome-only — the strongest causal evidence that the gate produces genuine evidence dependence rather than better-formatted mimicry.
Training and evaluation signals are decoupled
Informative ablation hierarchy
Unusually candid reporting
Weaknesses
Proprietary judge dependence for all headline faithfulness numbers
Single backbone family
Accuracy-parity contrast untested
EC-F1 construction bias unquantified
Heavy, partially private reproduction stack
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
MedAgent-R1 (arXiv 2608.30676; Qwen2.5-7B backbone) targets the gap between answer accuracy and justification faithfulness in RL-trained medical retrieval agents. The central empirical claim is 'confident hallucination': moving from SFT to outcome-only GRPO raises average accuracy from 69.64% to 74.56% across MedQA/MedMCQA/PubMedQA/MMLU-Medical (Table 1) while citation fabrication doubles from 16.5% to 31.8% and EC-F1 falls 65.1→58.7 (Table 2). The constructive claim: a faithfulness-gated reward — racc·1[rfaith>0] plus 0.2·rvalid, 0.5·rfaith, 0.1·rconcise (Eq. 3) — cuts fabrication to 4.7%, lifts EC-F1 to 82.6, holds accuracy at 75.12%, and adds 13.2 points of HealthBench Safety over outcome-only (Table 3). Versus GPT-4o under the identical agentic loop, it wins only the faithfulness-specific HRS dimensions (Factual Support 4.55 vs 4.25; Overclaiming 4.40 vs 4.15) while losing overall accuracy 75.12 vs 84.32 — a scope the abstract states correctly.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is 'confident hallucination' in MedAgent-R1?
A failure mode the paper identifies: an RL-trained retrieval agent answers correctly from its internal parametric memory while backfilling plausible-looking citations. Under outcome-only RL, accuracy rose from 69.64% to 74.56% but citation fabrication rose from 16.5% to 31.8%, meaning justifications cite evidence that does not actually support them.
What results does MedAgent-R1 achieve?
On the physician-annotated MedFaith-Eval (800 questions), it cuts citation fabrication to 4.7%, raises evidence completeness (EC-F1) to 82.6, keeps accuracy at 75.12% across four medical QA benchmarks, and gains 13.2 points on a 1,000-consultation HealthBench safety subset versus its outcome-only trained variant.
Does MedAgent-R1 beat GPT-4o?
Not overall: GPT-4o with the same agentic retrieval loop scores 84.32% accuracy versus 75.12%. MedAgent-R1 outscores GPT-4o only on the faithfulness-specific judge dimensions Factual Support (4.55 vs 4.25) and Overclaiming (4.40 vs 4.15).
Is code or data available for MedAgent-R1?
The paper states that judge prompts, evaluation scripts, and raw judge outputs are released so the evaluation can be reproduced. The provided text does not state a release of training code, teacher trajectories, or the MedFaith-Eval annotations.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The excerpt is a sampled extraction: middle sections marked [...sampling gap...] were skipped, including parts of §3.2 reward definitions and Appendix Q (cross-judge validation); judgments touching those rest on in-text references only.
- Whether training code and MedFaith-Eval annotations are released is not stated in the provided text; only judge prompts, evaluation scripts, and raw judge outputs are explicitly promised (Appendix A).
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.