Semantic-TVM: Structure-Preserving Trustworthy Virtual Memory for Memory-Augmented and Tool-Using Agents
Yu Li, Qikun Cai, Tao Huang, Chen Hou
cs.AI
Quick verdict
Claims span-level memory projection recovers masked-agent utility (84% vs 52% task success); promising closed-loop design, but small eval, no stats, no code.
Summary
The paper addresses privacy exposure in memory-augmented, tool-using LLM agents, whose serialized requests to remote providers can contain exact private values from retrieved memory and tool observations. It proposes Trustworthy Virtual Memory (TVM), a local runtime that separates the remote-visible context from locally held exact-value state: retrieved memory is projected into an online-safe view, tool actions are resolved locally against stored bindings, and observations are re-projected before each subsequent request. Two granularities share this closed loop: Rule-TVM replaces whole protected fields with recoverable handles; Semantic-TVM replaces only sensitive spans predicted by a trusted local model (a locally served gemma4:e4b-it-qat at temperature 0), preserving task-relevant context. On Memory-EHR (MIMIC-III, EHRAgent-style) and Memory-RAP (WebShop) with DeepSeek-V4-Flash and MiniMax-M2, both variants cut leak rates from 78.5–100% under No Defense to at most 1% on EHR and 12% on RAP, while Semantic-TVM raises EHR Task Success to 84.17% (DeepSeek) and 76.83% (MiniMax) versus 52.33% and 6.17% under Rule-TVM. An ablation shows span granularity, not the classifier alone, drives recovery. Limitations: exposure is exact-match against a manifest only, incomplete trials are excluded from leak denominators, no significance tests across three seeds, an 11–14 point gap to No Defense remains, RAP lacks ground truth, and no code or supplementary failure data is provided.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A trusted local runtime that projects memory, resolves handles, and reprojects observations can reduce detectable registered-value exposure from 78.5–100% to at most 1% (EHR) and 12% (RAP) across full multi-turn agent workflows.
- Span-level projection (Semantic-TVM) recovers most of the EHR task utility lost under whole-field replacement: Task Success 84.17% vs 52.33% on DeepSeek and 76.83% vs 6.17% on MiniMax, at leak rates of at most 1%.
- The utility recovery stems from projection granularity itself, not from replacing deterministic rules with a learned model — Classifier-TVM stays near Rule-TVM (47.83%/7.00%).
- On Memory-RAP both TVM granularities keep execution success above 99% while cutting exposure from 100% to at most 12%.
Strengths
Closed-loop design addresses a real failure mode of one-shot masking
Unlike the PAPILLON-style baseline (ref [5]), TVM's resolve-then-reproject loop (Sec. 2.1, Eq. 2) handles protected values that re-enter later requests via trusted tool observations, and validated resolution restores exact identifiers only inside the local executor — the actual reason masking breaks in multi-turn workflows.
Ablations cleanly isolate the source of utility recovery
Classifier-TVM (RQ2) keeps Rule-TVM's all-or-nothing operation but swaps in a learned field classifier, landing at 47.83%/7.00% Task Success vs Rule-TVM's 52.33%/6.17% — showing span granularity itself, not the use of a local model, drives the jump to 84.17%/76.83%. Table 2 further decomposes: Recovery lifts EHR Task Success 1.67→83.17; the scene Adapter cuts RAP Leak Rate 88.00→8.00.
Honest, explicitly scoped threat model
Multi-axis, multi-provider evaluation with consistent numbers
Weaknesses
No statistical inference anywhere
Leak-rate denominator conditions on unreported capture success
Residual utility cost vs No Defense is underplayed
Reproducibility gaps
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This method paper (arXiv 2609.15011, cs.AI, Sept 2026, conference-short format) targets a concrete gap in LLM-agent privacy: one-shot masking (the PAPILLON-style baseline, ref [5]) protects the initial request but fails across a multi-turn workflow, because trusted tool observations re-introduce protected values and irreversible masking removes identifiers that tools need. The authors propose Trustworthy Virtual Memory (TVM), a local mediator with three coupled operations — projection (Eq. 1), validated handle resolution, and observation reprojection (Eq. 2) — instantiated at two granularities: Rule-TVM (whole-field handle replacement) and Semantic-TVM (span replacement predicted by a local model). Central claims: (1) detectable registered-value exposure drops from 78.5–100% under No Defense to at most 1% (EHR) and 12% (RAP); (2) span granularity recovers most EHR utility lost to whole-field replacement (Task Success 84.17% vs 52.33% on DeepSeek; 76.83% vs 6.17% on MiniMax); (3) recovery comes from granularity itself, not from using a local model (Classifier-TVM: 47.83%/7.00%).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Semantic-TVM in one sentence?
It is a local runtime that replaces only the sensitive spans in an LLM agent's retrieved memory with recoverable placeholders before sending context to a remote model, keeping exact values on-device while preserving the surrounding task-relevant context that whole-field masking would destroy.
How much utility does Semantic-TVM recover over whole-field replacement?
On the Memory-EHR benchmark, Task Success rises from 52.33% to 84.17% with DeepSeek and from 6.17% to 76.83% with MiniMax, while leak rates stay at or below 1%; an ablation (Classifier-TVM at 47.83%/7.00%) shows the gain comes from span-level granularity, not from using a learned model per se.
Does TVM guarantee privacy?
No. The paper measures only exact or conservatively normalized matches of values listed in a protected-value manifest within captured outgoing requests. Semantic inference from retained context, handle cardinality and co-reference patterns, and values reappearing in unanticipated structures are explicitly acknowledged as out of scope (Sec. 5).
Is code or data available for this paper?
The provided paper text contains no code or data release statement. Supplementary material with failure-stage counts is referenced but not included; the benchmarks derive from access-restricted MIMIC-III and the public WebShop environment.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Fig. 1 (architecture) and Fig. 2 (bar chart) are available only as embedded text values; visual details could not be verified.
- Supplementary material with failure-stage counts is referenced but not part of the extracted text.
- Per-configuration trial counts and Capture Complete values are absent from the main text, limiting statistical assessment.
- Preprint (arXiv 2609.15011, Sep 2026); peer-review status and venue are unknown.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.