SpliTEE: Improving LLM Inference on Trusted Hardware with Differentially Private GPU Outsourcing
Shashie Dilhara Batan Arachchige, Robin Carpentier, Hassan Jameel Asghar, Dali Kaafar
cs.CR · cs.AI · cs.LG
Quick verdict
Extends Slalom TEE-GPU split LLM inference with DP noise, not encryption; strong theory and attack eval, but privacy accounting skips offload composition.
Summary
The paper extends Slalom's TEE–GPU split-inference architecture to LLMs, replacing encryption with differential privacy. Linear projections (Q/K/V, O, Gate/Up, LM head) are outsourced to an untrusted GPU; their inputs are masked with Gaussian noise calibrated to the global ℓ2-sensitivity of the preceding component (RMSNorm, softmax attention, SiLU-gated MLP), for which the authors derive worst-case bounds. Inside an Intel TDX guest, precomputed corrections Wη are subtracted to recover the true matrix product, keeping all arithmetic in floating point and avoiding Slalom-style quantization; a floating-point error analysis bounds cancellation error as a function of ε (mean bound ≈4.3× the observed error, though ~10% of evaluated elements exceed the bound). On Llama-3.2-3B and Qwen3-4B, split inference is roughly twice as fast as full-TDX CPU inference and 5–15 seconds faster than a Slalom-adapted baseline; at ε=10 all 100 Dolly-prompt greedy outputs match the GPU baseline exactly, while ε=1 yields 84% exact match and 99.67% semantic similarity. A trained reconstruction attacker recovers ~80% semantic similarity from unmasked activations, but only ~5% under ε=1, below the ~20.7% token-F1 overlap between unrelated prompts. Main limitations: the ε-GDP guarantee is stated per offload and is not composed over the many offloads per prompt; the GPU is only honest-but-curious with no result verification; evaluation covers small models and 100–500 prompts.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Unmasked intermediate representations sent to the GPU leak prompts: a trained inversion attack reconstructs prompts at ~80% semantic similarity (79.87% QKV-only, 82.19% composite; Table 6).
- Calibrated Gaussian masking with ε=1 drops reconstruction to ~5% semantic similarity, below the ~20.7% token-F1 overlap between unrelated prompts, even for noise-aware attackers with full knowledge of the mechanism.
- The split scheme runs ~2× faster than full-TDX CPU inference and 5–15 s faster than a Slalom-adapted encryption baseline on Llama-3.2-3B and Qwen3-4B, while remaining output-faithful (ε=10: 100% exact greedy match on 100 Dolly prompts).
- Floating-point cancellation error from masking/correction admits a closed-form upper bound (Theorem 4, Eq. 27) that tracks observed error and lets ε be chosen so accuracy is unaffected.
Strengths
Rigorous global sensitivity analysis as a reusable contribution
Section 5 (Props. 4–8, Theorems 1–3) derives worst-case ℓ2-sensitivity bounds for RMSNorm outputs, softmax attention and the SiLU-gated MLP (e.g., Δact ≤ 2√d·(√d‖α‖₄+‖β‖₂)²·‖Wup‖·‖Wgate‖), applicable to any system outsourcing LLM linear ops, not just this architecture.
Floating-point theory is validated against measurement, not just stated
Theorem 4 / Eq. (27) bound cancellation error as a function of ε; Table 4 shows the mean bound tracks observed error at a stable ≈4.3× ratio across all ε values, and §7.3.2's controlled-error experiment (Table 5) maps error magnitudes to semantic-similarity collapse (99.20% at 2.06×10⁻⁴ vs 9.91% at 2×10⁻²), giving practitioners a usable ε-selection procedure.
Adaptive, well-controlled attack evaluation
Quantified engineering trade-offs rather than hand-waving
Reproducible implementation detail
Weaknesses
Privacy guarantee is not composed over repeated offloads
The derived error bound is violated in ~10% of cases in practice
Strictly weaker guarantees than Slalom on integrity
Utility is measured as self-similarity, not task performance
Noise-bank scaling is unquantified
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
SpliTEE (arXiv 2609.15039, cs.CR; Macquarie University) targets prompt confidentiality for LLM serving split between a trusted execution environment (Intel TDX) and an untrusted GPU. Building on Slalom (Tramèr & Boneh 2019), it outsources linear projections (Q/K/V, O, Gate/Up, LM head) to the GPU but masks their inputs with Gaussian noise calibrated by ε-GDP instead of finite-field stream-cipher encryption, avoiding quantization entirely. Four claims anchor the paper: (1) unmasked intermediates leak prompts — a reconstruction attack reaches ~80% semantic similarity (§7.4, Table 6); (2) a global sensitivity analysis bounds the required noise (§5); (3) a floating-point error bound (Theorem 4, Eq. 27) governs accuracy as a function of ε (§5.5, §7.3.2); (4) the system is ~2× faster than full TDX-CPU inference and 5–15 s faster than a Slalom adaptation at equal or better fidelity (§7.1–7.3).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Why not just run the whole LLM inside the TEE?
CPU-only TEEs are far slower than GPUs for LLM inference. The paper measures 53–83 seconds for up to 800 generated tokens inside an Intel TDX virtual machine versus 3–5 seconds on a GPU. SpliTEE therefore keeps only non-linear operations in the TEE and outsources matrix multiplications to the GPU with masked inputs.
How is SpliTEE different from Slalom?
Slalom (Tramèr & Boneh, 2019) encrypts the inputs outsourced to the GPU with stream ciphers over finite fields, which forces quantization and accuracy loss — in this paper's LLM adaptation, 0% of generated responses exactly matched the baseline. SpliTEE instead adds differentially private Gaussian noise, keeping all arithmetic in floating point; accuracy is tunable via the privacy parameter ε, and it ran 5–15 seconds faster on the tested LLMs.
Does the GPU really learn nothing about my prompt?
Under ε=1 masking, a strong reconstruction attacker trained with full knowledge of the model and the noise mechanism recovered prompts at only ~5% semantic similarity — below the ~20.7% token overlap that naturally occurs between unrelated prompts. Caveat: the formal ε-GDP guarantee is stated per outsourced operation, not composed across the hundreds of offloads in a full request, so the practical protection rests largely on these attack experiments.
Is the code and data available?
Yes. The authors state the implementation is public at github.com/DPVault/SpliTEE. The evaluated models (Llama-3.2-3B-Instruct, Qwen3-4B-Instruct) are public on Hugging Face, and prompts come from the open Databricks Dolly dataset.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The extracted text has omitted passages: exact Table 1 timing values and the statements of Theorems 1–2 (§5.2–5.3 proofs) were not visible; timing claims are taken from prose (§4.3, §7).
- Figures (architecture Fig. 2, outsourcing Fig. 3, timing curves Figs. 4–5, attacker model Fig. 6) could not be analyzed from text.
- Section 7.4.1–7.4.2 (attacker architecture and training details) is only partially present, so attack-implementation reproducibility is judged from the visible subsections.
- Preprint (arXiv v1, 14 Sep 2026), not yet peer-reviewed; internal date printed 15 Sep 2026.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.