Confuse the Model, Control the Flow: Understanding and Mitigating Privacy Leakage from LLM Agents with Information Flow Control
Minsun Shim, Ramisha Raida Karim, Ruthwik Jakkula, Kaiwen Zhou, Xin Liu, Xin Eric Wang, Zhou Li
cs.CR · cs.AI
Quick verdict
Shows LLM-agent privacy defenses fail when enforcement shares the attack surface; three attacks plus an IFC interceptor cut leaks to near zero. Credible.
Summary
The paper argues that LLM-agent privacy defenses are structurally weak whenever enforcement is a judgment the same model makes over the same adversary-influenced context it must govern. The authors introduce three injection-free attacks: Collaborative Workspace Lure (extraction disguised as collaborative editing), Semantic Obfuscation Attack (real records revealed by refusing to delete fabricated ones), and Channel Decoupling Attack (request and disclosure split across channels). Against the SPR-D2 defense these reach 43.8–75.6% item-level leakage versus 1.9–3.0% for SPR's own attacks. The proposed defense, FLOWSEAL, moves enforcement outside the LLM's context: provenance-based ownership labels, a mandatory tool-call interceptor implementing an information-flow-control lattice with controlled declassification, plus a cooperative consent API. Across three benchmarks (SPR, PrivacyLens, ConVerse), five defense baselines, eight attacks, and three LLM backends, it holds item-level leakage to 0–3.3% at 72.4% utility versus 75.9% for the best baseline; on a live MCP Gmail/Notion deployment, SOA leakage drops from 34.7% under PrivacyChecker to 0%. Limitations acknowledged include paraphrase evasion and benign over-blocking in the content checker, residual LLM-dependent declassification, and modestly higher cost ($0.052 vs $0.042 per run); leak metrics rest on LLM judging without reported human validation.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Prompt-based agent privacy defenses are structurally vulnerable because the LLM that decides whether enforcement applies operates on the same adversary-controlled context it must govern.
- Three new injection-free attacks (CWL, SOA, CDA) achieve 43.8–75.6% item-level leak rates against SPR-D2, versus 1.9–3.0% for the SPR-discovered attacks those defenses were built to withstand.
- FLOWSEAL's tool-level interceptor with provenance labels and an IFC lattice reduces item-level leakage to 0–3.3% while retaining 72.4% benign-task utility (best baseline 75.9%), stably across three LLM backends.
- The attack and defense transfer beyond simulation: on a live MCP deployment with Gmail and Notion, SOA leakage falls from 34.7% under PrivacyChecker to 0% under FLOWSEAL.
Strengths
Structurally motivated threat analysis, not another prompt tweak
Section I articulates a crisp failure principle — enforcement and attack surface coincide when the LLM judges privacy over adversary-controlled context — and quantifies it: CWL/SOA/CDA reach 43.8–75.6% item-level leakage on SPR-D2 versus 1.9–3.0% for SPR's own discovered attacks.
Mechanistically distinct injection-free attacks
CWL (collaborative reframing of extraction as shared-artifact editing), SOA (omission side channel: refusing to delete fabricated records reveals which are real), and CDA (cross-channel state separation defeating session-scoped consent 'by construction') require no prompt injection or tool compromise, a regime clearly separated from EIA/LeakAgent-style work in Section II.
Defense places enforcement where the adversary cannot reach
Unusually broad and partly live evaluation
Candid failure-mode and cost accounting
Weaknesses
All leak metrics are LLM-judged with no reported human validation
No inferential statistics for the headline comparisons
The formal guarantee is narrower than the conclusion's wording
Utility evaluation is author-constructed and the trade-off is mildly understated
Attack fidelity varies across benchmarks
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper (arXiv:2609.14003, cs.CR/cs.AI; UC Irvine with UC Santa Cruz, Davis, Santa Barbara) studies contextual privacy in personal LLM agents that hold a user's private data and interact with external parties. It makes four claims: (1) prompt/training/consent-based defenses share a structural weakness — enforcement is an LLM judgment made over the same adversary-controlled context it is meant to govern; (2) three injection-free attacks (Collaborative Workspace Lure, Semantic Obfuscation Attack, Channel Decoupling Attack) exploit this; (3) FLOWSEAL, a defense placing enforcement in a tool-level interceptor outside the LLM's context, grounded in data provenance and an information-flow-control lattice with controlled declassification, restores robust confidentiality; (4) it achieves near-zero leak rates at modest utility cost, stable across backends, including a live MCP deployment. The two load-bearing empirical anchors are the SPR-D2 contrast (43.8–75.6% vs 1.9–3.0% item-level leak rate, Section I) and the live Gmail/Notion SOA result (34.7% under PrivacyChecker → 0%, abstract and Section VIII).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is FLOWSEAL?
FLOWSEAL is a defense for personal LLM agents that intercepts every tool write and share call outside the LLM's conversational context. It tags sensitive records with ownership labels taken from data provenance (email sender fields, document ACLs), enforces an information-flow-control lattice (blocked or allowed destinations), and uses an isolated LLM check only for content declassification decisions. The paper reports it cuts item-level leakage to 0–3.3% while keeping 72.4% task utility.
Why do prompt-based privacy defenses fail against these attacks?
Because the enforcement decision is itself an LLM judgment made over the same conversation the attacker shapes. If an attack disguises extraction as ordinary work (a shared document to fill), the model never recognizes a privacy decision, so every downstream check is skipped. On the SPR-D2 defense, the paper's three attacks leak 43.8–75.6% of sensitive items, versus 1.9–3.0% for the attacks those defenses were designed against.
What are the three new attacks (CWL, SOA, CDA)?
Collaborative Workspace Lure frames data extraction as collaborative editing of a shared artifact, so the agent does not classify it as an information request. Semantic Obfuscation Attack asks the agent to clean a mix of genuine and fabricated records; refusing to delete some entries reveals which records are real. Channel Decoupling Attack sends the extraction request over one channel (email) and collects the disclosure over another (Notion), so session-level consent tracking cannot connect the two.
Does FLOWSEAL hurt task performance?
Modestly, per the paper: benign-task utility is 72.4% under FLOWSEAL versus 75.9% for the best prompt-based baseline, at roughly $0.052 per run (vs $0.042 for PrivacyChecker) and 37% more latency than running no defense. The authors also acknowledge false positives where benign writes that merely mention a protected record by name get blocked.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The extraction is truncated: parts of Section IV (attack details), the Section VI results tables, and the Section VII ablations are missing, so several numbers were cross-checked from abstract, intro, and prose rather than read directly from tables.
- Figures are not analyzable from text (lattice diagram, attack scenarios Fig. 3–4, defense traces); Table II/VII values are inferred from surrounding prose.
- Metric definitions (Appendix G) and the D3/A3 prompt details (Appendix F) are not visible, limiting assessment of judge prompts and attack-search rigor.
- arXiv preprint (Sep 2026), not yet peer-reviewed; the artifact is promised but not yet verifiable.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.