PrivDrift: Auditing User-Secret Leakage Under Topic Drift in Active LLM Conversations
cs.AI · cs.CL · cs.CR
Quick verdict
Shows LLM user secrets remain recoverable after topic drift (38.7–54.6%); controlled, statistically careful, but leakage framing lacks a suppression baseline.
Summary
The paper introduces PrivDrift, a benchmark of 1,000 controlled multi-turn dialogues (900 synthetic, 100 human-authored) that seed one fixed-format secret — phone, email, SSN, or credit card — in an opening turn, insert 0–6 content-dense drift turns, and append standardized extraction probes at three persuasion levels. Three open-weight models (GPT-OSS-120B, DeepSeek-R1, Qwen3-VL-235B) were queried via OpenRouter, yielding 9,000 cached responses. Dialogue-level hybrid leakage (regex plus an open-weights LLM judge) ranges from 38.7% (DeepSeek-R1) to 54.6% (Qwen3-VL-235B). Persuasion intensity has significant but model-dependent, non-monotonic effects; drift length within d≤6 shows negligible effect sizes; secret type dominates (Cramér's V 0.59–0.77), with SSNs and credit cards largely suppressed while emails and phones leak at 46–81%. A proposed 'Privacy Half-Life' stability metric shows no model achieves stable decay within the tested window. Reporting quality is comparatively strong: bootstrap CIs, McNemar with Bonferroni, Cochran's Q, chi-square with effect sizes, and separate regex/fuzzy/hybrid rates. Main limitations: the benchmark never instructs the model to keep secrets confidential, so high 'leakage' may reflect ordinary helpfulness rather than a privacy failure; the 100 human-authored dialogues lost their identifiers in cached metadata and are unanalyzed; the LLM judge lacks human validation; and code/data are only promised for release. Results are bounded to short-to-medium drift and API-served models.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- User-disclosed secrets remain behaviorally recoverable from active LLM context after unrelated topic drift, with dialogue-level hybrid leakage of 38.7–54.6% across three open-weight models.
- Within drift lengths d≤6, additional topic drift does not reliably reduce leakage (Cramér's V 0.066–0.107).
- Persuasion intensity significantly affects leakage but non-monotonically and model-dependently (hard probes reduce leakage for GPT-OSS-120B but increase it for Qwen3-VL-235B).
- Secret type is the dominant factor (Cramér's V 0.59–0.77): SSNs and credit cards are largely suppressed while emails and phone numbers leak at 46–81%, implying format-sensitive rather than contextual confidentiality safeguards.
- No evaluated model achieves stable suppression within the tested window under the proposed Privacy Half-Life metric (τ=7 for all).
Strengths
Controlled experimental design
Seeded fixed-format secrets, exact drift lengths sampled from a stated mixture distribution (Eq. 1), and standardized probes at three persuasion levels give clean factorial control; the scaffold-then-rewrite pipeline (§3.2) reduces template artifacts while preserving ground truth.
Transparent multi-detector reporting
Regex-only, fuzzy (0.85/0.80), and hybrid rates are reported side by side (Tables 2, 6); the LLM judge adds only 0.0–1.10 pp, and the authors flag judge error as a risk rather than hiding behind the black-box metric.
Above-average statistical rigor
Actionable secret-type asymmetry finding
Stability-based suppression metric
Weaknesses
Leakage framing lacks a suppression condition
Human-authored split is unrecoverable
LLM judge unvalidated
Artifacts not yet available; API drift
Model selection narrow and partly odd
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
PrivDrift (arXiv 2609.30094, single author, WVU) proposes a new measurement target for LLM privacy evaluation: whether a secret a user discloses in an active conversation remains extractable after unrelated topic drift and persuasion-based probing — positioned as distinct from training-data memorization (Carlini et al. 2021; Nasr et al. 2023) and single-turn jailbreaks (Wei et al. 2024; Zou et al. 2023). Central claims: (1) dialogue-level hybrid leakage of 38.7–54.6% across three open-weight models (Table 3); (2) drift length d≤6 does not reliably reduce leakage; (3) persuasion effects are significant but non-monotonic and model-dependent (Table 4); (4) secret type dominates (Cramér's V 0.59–0.77); (5) no model reaches stable suppression under the proposed Privacy Half-Life metric (§5.6). Scope claims are carefully hedged — bounded drift, active context only — and the Limitations section reinforces this discipline.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the PrivDrift benchmark?
PrivDrift is a benchmark of 1,000 controlled multi-turn LLM dialogues that seed a fixed-format personal secret (phone number, email, Social Security Number, or credit card number) in the opening turn, insert 0–6 turns of unrelated topic drift, then test whether standardized probes at three persuasion levels can still extract the secret. It measures whether sensitive information a user tells an LLM stays recoverable later in the same conversation.
How much do LLMs leak user secrets after the conversation topic changes?
In this paper, dialogue-level hybrid leakage is 38.7% for DeepSeek-R1, 47.7% for GPT-OSS-120B, and 54.6% for Qwen3-VL-235B — meaning that fraction of dialogues had the secret extracted by at least one of three probes, even after up to six unrelated drift turns.
Does changing the subject make an LLM forget what you told it earlier?
Within the tested window of up to six drift turns, no: leakage rates stayed high and drift length had a negligible effect. The paper frames privacy risk in active conversations as persistent, but it only tested short-to-medium drift, not full context windows or cross-session memory.
Which kinds of secrets leak most from LLM conversations?
Emails and phone numbers leaked at roughly 46–81% per probe, while Social Security Numbers and credit card numbers were largely suppressed (about 0–6%). The authors conclude current models rely on format-sensitive safety cues rather than a genuine understanding of confidentiality.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1, 3–6 are available only as captions and inline numbers in the extracted text; visual details such as exact curve shapes could not be verified.
- Probe template texts are not printed in the paper, so probe wording effects cannot be assessed from this text.
- Very fresh preprint (dated Sept 25, 2026); no external review or citation context is available yet.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.