Separating Capability from Confidence: Grounded Dual-State Calibration for GRPO-Trained Medical Vision-Language Models
Yangyang Xie, Ke Hao, Jiaqi Liu, Yun Gu, Xinglin Zhang
cs.CV · cs.LG
Quick verdict
DualRead post-hoc reads confidence from a frozen GRPO medical VLM's internal states; CCG-AUC tests grounding; strong ID gains, uneven OOD vs baselines.
Summary
The paper argues that jointly optimizing verbalized confidence with GRPO distorts the answer policy and collapses confidence toward near-binary values, and proposes DualRead: freeze the GRPO-trained actor, then estimate answer reliability from its internal states via one teacher-forced replay. A Pre-Answer Solvability Read (MLP on the last prompt position) and a Post-Answer Evidence Read (linear head on the answer state plus a residual from answer-conditioned cosine pooling over image-token states) are temperature-scaled and fused by equal-weight log-odds averaging. The authors also introduce CCG-AUC: each image is replaced by a real same-question hard negative with a different reference answer, and the metric scores whether confidence drops when the actor flips from correct to incorrect; a Strict variant holds the canonical answer fixed. On SLAKE-test, DualRead reaches AUROC .952 (Qwen3-VL-2B) and .947 (MedGemma1.5-4B) versus .822/.771 for RLCR and roughly chance (.52/.51) for the same actor's verbalized confidence, with accuracy unchanged. Across six OOD datasets (macro-averaged), gains are clear on MedGemma but weaker on Qwen, where the untrained Base model's verbalized confidence actually posts higher OOD CCG-AUC (.782 vs .624); supplementary tables candidly report per-domain losses. Limitations: requires internal-state access and per-actor reader fitting, small decisive sets inflate OOD CCG variance, and no multiple-comparison correction is described.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Jointly optimizing verbalized confidence within GRPO interferes with answer learning (RLCR SLAKE accuracy .608 vs .639 for capability-only RLVR on Qwen) and collapses confidence toward near-binary values (same-actor verbalized AUROC ≈ .516/.503).
- Reading a frozen GRPO actor's internal states (PSR + PER, equal-weight log-odds fusion) yields far better correctness discrimination (SLAKE AUROC .952/.947) than its verbalized confidence while leaving answers and accuracy exactly unchanged.
- CCG-AUC, built on real same-question hard-negative images restricted to correct→incorrect flips, measures whether confidence responds to answer-decisive visual evidence; DualRead posts the best ID Natural CCG-AUC (.784/.819) among same-actor methods.
- OOD reliability improvements (six datasets, macro-averaged) hold clearly on MedGemma1.5-4B but are weaker on Qwen3-VL-2B, where the paper's own tables show the untrained Base verbalized confidence achieving higher OOD CCG-AUC.
Strengths
Genuine decoupling with verifiable invariance
Confidence is learned entirely outside policy optimization (grad through the frozen actor is zero), and every shared-actor row in Tables 1/8/9 reports byte-identical accuracy (SLAKE .639±.009 Qwen, .677±.011 MedGemma), cleanly isolating the confidence interface from answer changes — a stronger control than most calibration papers provide.
Well-constructed CCG-AUC metric
Real in-distribution hard negatives (same question, different reference answer) avoid the novelty-detection confound of blanked/shuffled images; restriction to the actor-defined decisive set (correct→incorrect flips, |E|≈110, VBR .248/.251 in Table 11b) plus the Strict variant fixing the canonical answer addresses answer-wording artifacts; the rule is parameter-, scale-, and model-free with explicit preconditions (Supp. F).
Unusually rigorous evaluation protocol
Broad, carefully implemented baseline field
Candid negative reporting
Weaknesses
OOD grounding advantage does not hold against the untrained Base on Qwen
CCG-AUC power and cross-actor comparability limits
Potential ID familiarity channel in reader fitting
Fusion gains over PER alone are metric-dependent
White-box-only deployment
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper targets a concrete deployment failure: after GRPO post-training, medical VLMs verbalize near-binary, poorly calibrated confidence (same-actor verbalized AUROC .516/.503 on SLAKE, Table 1), and policy-level fixes (RLCR, DCPO, VL-Calibration) entangle confidence with the answer gradient. DualRead instead freezes the actor and reads reliability from internal states: one teacher-forced replay yields the pre-answer state (PSR, an MLP on the last prompt position) and the post-answer state (PER = P-State linear head plus a P-Visual residual built from answer-conditioned cosine pooling over image-token states, κ=0.1). Branches are temperature-scaled and fused by equal-weight log-odds averaging. The companion metric CCG-AUC replaces each image with a real same-question hard negative (different reference answer) and scores whether confidence drops on the actor-defined decisive set where the answer flips correct→incorrect; a Strict variant holds the canonical answer fixed. Three testable claims: (1) joint confidence optimization harms answers and collapses confidence; (2) state-read confidence beats verbalized confidence at unchanged accuracy; (3) CCG-AUC exposes whether confidence tracks visual evidence.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is DualRead?
DualRead is a post-hoc confidence estimator for GRPO-trained medical vision-language models: the RL-trained answer model is frozen, and small learned probes read its internal hidden states immediately before and after answer generation to output a calibrated probability that the answer is correct — without changing any answers.
What does CCG-AUC measure?
CCG-AUC (Counterfactual Confidence Grounding AUC) checks whether a model's confidence is grounded in the image: each image is replaced by a real image that has a different reference answer for the same question, and the metric scores how often confidence drops on cases where the model's answer flips from correct to incorrect. A score of 0.5 means no directional response; 1.0 means confidence always decreases.
Does DualRead change the model's answers or accuracy?
No. The actor is frozen and confidence is computed from internal states after generation, so reported accuracy is identical to the underlying GRPO actor (SLAKE .639 for Qwen3-VL-2B and .677 for MedGemma1.5-4B).
What are DualRead's main limitations?
It requires white-box access to hidden states and a probe trained per actor (unusable with API-only models), and its out-of-distribution grounding advantage is uneven: on Qwen3-VL-2B the untrained base model's verbalized confidence achieves higher OOD CCG-AUC than DualRead, per the paper's own tables.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The excerpt contains two '[...sampling gap...]' regions (mid-method/analysis and parts of supplement D, F, G, H plus table tails); judgments about those parts rest only on the surrounding visible text.
- Table 3 (SC@K/semantic-uncertainty cost comparison) and Table 4 (full hyperparameter list) fall inside sampling gaps; their values were not verified.
- Column alignment of the GRPO(RLVR) row in the extracted Table 1 is ambiguous; that row's values were cross-checked against Table 8 instead.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.