Investigating Linear Probe Robustness to Linguistic Register, Medical Specialty, and Corpus Shifts in Medical QA
Nishant Mishra, Ameen Abu-Hanna, Iacer Calixto
cs.CL · cs.LG
Quick verdict
Paper claims LLM truth directions survive register and specialty shifts but break across corpora (−0.21 AUROC on MedMCQA); well-controlled benchmark study.
Summary
The paper asks whether the 'truth direction' that linear probes recover from LLM hidden states survives realistic input changes, arguing that prior disagreement arises because cross-dataset transfer confounds several shifts at once. The authors build a controlled benchmark: 500 MedQA facts rewritten by Claude Sonnet 4.5 into four registers (textbook, patient, clinical-note, colloquial) with both answer polarities (4,000 variants), specialty labels from S-MedQA, and transfer sets spanning MedMCQA, MMLU-medical, reformatted MedMCQA, and 84 human-written MedRedQA questions. Probing four open-weight LLMs (2–8B) with L2 logistic regression at every second layer, they find a mean register gap of 0.095 AUROC (−0.064 to −0.126 per model) and specialty gap of 0.031, versus 0.21 for MedQA→MedMCQA and 0.12 for MMLU-medical; a format control (reformatted MedMCQA, 0.559 vs 0.561) excludes question style, pointing by elimination to distractor authorship. The register result replicates with a Gemini generator and holds on human-written questions for the two larger models (Gemma-2-2B falls to 0.542). MLP probes add only 0.037; a difference-of-means probe matches. The probe beats entropy/self-consistency baselines but P(True) ties it on the larger models; raw scores are severely miscalibrated (mean ECE 0.341, 0.135 after Platt). Limitations include LLM-generated evaluation text, small held-out sets (n=200), and per-cell best-layer selection, partly mitigated by sensitivity analyses.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- The linear truth direction in 2–8B LLMs is largely robust to writing-register shifts (mean Δ 0.095 AUROC on held-out facts) and to medical-specialty shifts (Δ 0.031) within the medical domain.
- The same direction degrades unevenly across corpora — 0.12 AUROC on MMLU-medical and 0.21 on MedQA→MedMCQA — and question format does not explain the break, implicating dataset construction, most plausibly distractor authorship.
- The signal is dominantly linear: an unregularized difference-of-means probe matches the logistic probe and a one-hidden-layer MLP adds only +0.037 AUROC.
- The probe outperforms entropy and self-consistency baselines but only ties P(True) on the larger models, so its practical edge is availability before generation; raw scores are severely miscalibrated (mean ECE 0.341).
Strengths
Factorial isolation of shift axes
Holding two of register/specialty/corpus fixed while varying the third (§2.1) directly resolves why prior cross-dataset transfer results conflict; the reformatting control (0.559 vs 0.561, §4.3) and worst-case additive bound (0.149 < 0.21) exclude question format and topic as the corpus-drop cause.
Protocol hygiene uncommon in probing work
Fact-level splits move both polarities together (Table 1); a label-permutation probe collapses to 0.488 mean AUROC (App. E.3); fidelity filtering moves the register gap by ≤0.0023 (App. B.4); a second generator reproduces the ordering (0.079 vs 0.095); the appendix quantifies that the earlier protocol understated Δregister by 0.042 (App. C.2).
Honest reporting of negatives
Strong reproducibility package
Actionable failure analysis
Weaknesses
Single-generator evaluation text
Optimistic layer selection as headline
Small evaluation sets under several claims
Corpus-drop mechanism untested directly
Narrow practical advantage over P(True)
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper targets a specific, well-motivated gap: proponents of linear 'truth probes' (Azaria & Mitchell 2023; Marks & Tegmark 2024; Bürger et al. 2024) and skeptics (Levinstein & Herrmann 2024; Haller et al. 2025; Orgad et al. 2025) disagree about generalization, but cross-dataset transfer experiments confound writing style, topic, distractor provenance, and difficulty. The authors decompose input shift into three controlled axes — register, specialty, corpus — and measure probe AUROC along each in isolation on medical QA. Central claims: (1) the truth direction is largely robust to register (mean Δ 0.095 AUROC on held-out facts, Table 2) and specialty (Δ 0.031 across five random 7/8-specialty partitions); (2) it degrades unevenly across corpora — 0.12 on MMLU-medical, 0.21 on MedQA→MedMCQA — with question format excluded via a reformatting control (0.559 vs 0.561, §4.3); (3) the signal is dominantly linear (MLP +0.037; diff-means matches); (4) probes beat entropy/self-consistency but tie P(True) on the two larger models, and raw scores are severely miscalibrated (mean ECE 0.341). Claims are scoped honestly in the visible discussion; the abstract's unqualified 'carries over to human-written patient questions' is the one place scope slips (see Risks).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is a 'truth direction' in an LLM?
It is a direction in the model's hidden-state space along which true and false statements separate, so a simple linear classifier (a probe) trained on hidden states can score whether a statement is correct from a single forward pass, without generating an answer.
Do truth probes generalize to new datasets?
According to this paper, within the medical domain they survive changes of writing style (about 0.10 AUROC drop) and medical specialty (0.03), but drop much more across corpora — 0.12 on MMLU-medical and 0.21 on MedQA→MedMCQA — indicating the signal is partly tied to how each dataset was built, plausibly its wrong-answer distractors.
Are these probes reliable enough for clinical safety monitoring?
No. The authors state that AUROC of 0.70–0.80 is insufficient for patient-facing safety, raw probe scores are badly calibrated (mean ECE 0.341; still 0.135 after Platt scaling), and the probe systematically rejects correct statements containing negation.
Is the code and benchmark data available?
Yes. The paper releases the 4,000-variant benchmark, evaluation-only sets, prompts, judge rubric, and full pipeline at github.com/mnishant2/MedProbe_release, with code under MIT and data under CC BY-NC 4.0.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text is a sampled excerpt with several [...sampling gap...] breaks: the conclusion/limitations section (§7, referenced in-text), parts of §3.4, and portions of Appendices B–C were skipped by sampling; assessments of those parts rely on cross-references in visible sections.
- Figures appear only as captions in the extracted text; figure-internal values could not be verified, so all numeric claims here come from the extracted tables.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.