Detokenization Leaks: Reconstructing Local LLM Outputs From Cache Traces
Roy Weiss, Benyamin Konstantinov, Eitam Sheetrit, Tomer Simon, Yisroel Mirsky
cs.CR · cs.AI
Quick verdict
Claims a universal cache side channel via LLM detokenization; rigorous hybrid F+R/P+P design and broad eval, but ASR rests on a lenient LLM judge.
Summary
This paper presents a cache side-channel attack that reconstructs the outputs of locally hosted LLMs by targeting detokenization. Unlike prior LLM cache attacks that assume shared model data, CPU offloading, or MoE architectures, it exploits the decode-table lookup that every default inference pipeline performs on the CPU. An unprivileged process co-scheduled on the SMT sibling of the inference thread uses Flush+Reload on shared tokenizer code (97.66% detection, zero false positives over 1.2M invocations) to time Prime+Probe snapshots of all 64 L1 cache sets during each token's decode. Noisy, collision-prone traces are clustered (K=64) into symbols, and fine-tuned Flan-T5-XL models reconstruct text from symbol sequences; profiling requires 250 targeted queries (~6 minutes) or roughly 7,650 benign queries. Across UltraChat, ChatDoctor, and Code-Alpaca, two inference frameworks, and three Intel CPUs, judge-based attack success reaches 56-87% on full UltraChat paragraphs, up to 92.85% on ChatDoctor, and 95.87% on code, but drops to 38.4% on a 13th-generation CPU and 30.12% in an end-to-end OpenClaw attack. Main limitations: success rates rest on a lenient GPT-4.1-mini judge that accepts partial or noisy matches; no statistical tests or run counts are reported; and the learned mapping does not survive process restarts.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Detokenization's decode-table lookup is a universal, default-on cache leakage source for local LLM pipelines, unlike prior attacks' assumptions of shared data, CPU offloading, or MoE architectures (§3.2, Table 4).
- Flush+Reload on shared tokenizer code provides a reliable per-token trigger (97.66% TPR, zero false positives over 1.2M invocations) that makes L1 Prime+Probe effective (§4.2).
- A clustering (K=64) plus Flan-T5-XL pipeline reconstructs text from noisy traces, reaching judged ASR up to 87.42% on full UltraChat paragraphs, 92.85% on ChatDoctor, and 95.87% on Code-Alpaca (§5.2, Tables 1-2).
- The attack transfers end-to-end to OpenClaw (30.12% ASR) and persists on a 13th-gen hybrid Intel CPU (38.40% ASR), with profiling achievable via 250 targeted queries in ~6 minutes (§5.4-5.6).
Strengths
Genuinely novel, deployment-universal leakage surface
Targets the CPU-side decode-table lookup performed in every default pipeline, avoiding prior works' assumptions of shared model data, CPU offloading, or MoE architectures (§3.2, Table 4); the authors state the decode table was not previously examined as a leakage source (§7).
Elegant two-channel synchronization design
Flush+Reload on shared tokenizer code gates L1 Prime+Probe timing, achieving 97.66% TPR with zero false positives over 1.2M invocations (§4.2); offline timing characterization (probe 8-23 µs after trigger) and randomized linked-list eviction sets in a modified CacheSC address C1/C2 concretely (§4.3).
Broad, honest evaluation matrix
Well-motivated reconstruction pipeline with ablation
Reproducibility and ethics posture
Weaknesses
Headline ASR rests on an unvalidated, lenient LLM judge
Thin statistical reporting
Deployment realism gap
Synthetic evaluation of the trigger and robustness
Qualitative-only mitigation analysis
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper (arXiv 2609.06674, cs.CR; BGU + Microsoft Security) claims the first cache side-channel attack that reconstructs LLM outputs under *default* local-deployment configurations, by targeting the CPU-side detokenizer rather than deployment-specific features (shared model data in [2, 18], MoE routing in [16]). Four contributions are stated: a universal leakage source (the decode-table lookup), Flush+Reload-based synchronization for Prime+Probe, a clustering-plus-language-model reconstruction pipeline, and an evaluation spanning datasets, frameworks, and hardware, capped by an end-to-end OpenClaw attack. The threat model is explicit: an unprivileged local process on the SMT sibling of a long-lived inference thread, needing only shared *code* pages. The paper is commendably explicit about the profiling requirement (§5.4), degradation on modern hybrid CPUs (§5.5), and the end-to-end performance drop (§5.6).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Can a CPU cache side channel really leak what a local LLM is writing?
This paper argues yes: the decode-table lookup performed during detokenization leaves token-dependent traces in the L1 cache. A demonstration shows an unprivileged program on the sibling hyperthread recording those traces and a trained language model reconstructing roughly 30-87% of outputs semantically, depending on the setup and how success is judged.
What hardware and access does the Detokenization Leaks attack require?
An SMT-enabled (Hyper-Threading) Intel x86 machine running a long-lived local LLM server such as Llama.cpp/Ollama or HuggingFace-based inference. The attacker runs as an ordinary user process pinned to the sibling logical core, needs no admin rights, no shared memory with the victim, and no GPU access — but must first profile the same process with 250 targeted or ~7,650 ordinary queries.
How well did the attack work against OpenClaw?
In the paper's end-to-end demo against the OpenClaw agent running Phi-3-mini locally, 30.12% of first response segments were judged semantically equivalent to the real output (mean embedding similarity 42.23%), down from 61.55% in controlled lab conditions — so it worked, with substantial degradation.
How can I protect my local LLM from this detokenization cache leak?
The paper's mitigations are all partial: disabling SMT blocks the L1 observation channel at a 25-35% performance cost, periodic process restarts invalidate the attacker's trained profile, and randomizing the decode table's memory layout raises re-profiling cost. No cheap complete defense is identified; the authors did not experimentally measure any mitigation.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Dense tables (Table 1, Table 6) were extracted with ambiguous alignment of the ± values to metric columns; quoted means were cross-checked against the §5.2 prose where possible.
- Figures 6, 7, and 9 are charts whose plotted points are only partially machine-readable; values cited from them are taken from the surrounding captions and prose, not from the raw data.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.