Exact Record Omission in Delta Attention: A Transport Criterion, Its Cost, and a Replay Certificate
cs.LG · cs.CL · cs.CR
Quick verdict
Derives an exact deletion criterion for delta-rule memory; receipts and masking leak, replay certifies exactly. Rigorous, but synthetic-only evaluation.
Summary
The paper asks whether deleting a record from an assistant's inference-time recurrent memory truly removes its influence, distinguishing behavioral suppression from a verified state change. Using the never-stored state — the state the same model would build from the conversation with the record omitted — the author unrolls the affine recurrence of delta-rule memory (Kimi Delta Attention) and proves an exact criterion: a saved boundary difference ('receipt') transported through later updates reaches the never-stored state if and only if the extra differences ('forcing') created by later updates cancel. On the 48B Kimi Linear hybrid, experiments use 40 pre-registered synthetic evaluation and 40 separate calibration contexts across eleven conditions. Attention masking suppressed all 40 exact greedy answers, yet a fixed three-query candidate-scoring attack reached AUC 0.740 at a measured 7.5% false-positive rate, and prompt-only forgetting still returned 33 of 40 targets; receipt transport left state residuals up to 32.9%. Checkpoint replay matched every audited array and the audit logits in all 80 contexts and preserved 120 retained answers. Controls include Mamba-2, Falcon-H1 and RWKV-7 with pre-recorded predictions, a 32-record Qwen3.5 cohort, and same-master bf16/8-bit precision comparisons. Limitations: synthetic contexts only, a fixed non-adaptive attack, frozen-trace replay, and small cohorts.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A boundary receipt transported through later delta-rule updates reaches the never-stored state if and only if the accumulated forcing created by later updates is exactly zero (Theorem 1).
- On 48B Kimi Linear, attention masking removes all 40 exact greedy target answers yet a fixed three-query candidate-scoring attack still distinguishes masked contexts (AUC 0.740, measured FPR 7.5%), while prompt-only forgetting returns 33 of 40 targets.
- Checkpoint replay exactly matches the never-stored state on every declared active array and audit logit in all 80 contexts while preserving all 120 retained answers.
- Native transport mismatch above measured arithmetic floors persists across recurrent families (Mamba-2, Falcon-H1, RWKV-7), a 32-record Qwen3.5 cohort, and same-master bf16/8-bit Kimi precisions.
- Replay also installs record replacements and supports sequential deletions with an intervening admission, matching fresh references in all 12 amendment contexts and 6 paired deletion orders.
Strengths
Exact theory with falsifiable, pre-recorded predictions
Theorem 1 gives an iff transport criterion via unrolling the affine recurrence; Corollary 1 classifies write rules, and the cross-family predictions for Mamba-2, Falcon-H1 and RWKV-7 were recorded before running the models (App. B.13) and all held (Fig. 4, Table 19; frozen-input errors 1.6–2.6×10⁻⁶).
Pre-registered prospective cohort design
40 evaluation + 40 calibration contexts frozen with a seed (2026091101), SHA-256 manifest hash, fixed prompts, decoding and analysis rules before any model call (App. A.1); attack threshold calibrated on never-stored calibration scores only (App. A.2).
Unusually honest statistical reporting
Layered artifact controls
Actionable cost engineering
Weaknesses
Entirely synthetic evaluation cohorts
The criterion cannot be evaluated cheaper than replay
Fixed, non-adaptive attack only
Frozen-trace certificate scope
Small cohorts and residual heterogeneity
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper tackles a narrow but well-posed question: when a user deletes a record from a conversation served by a model with recurrent (linear-attention/delta-rule) memory, has the memory actually changed? The author defines the exact reference — the *never-stored state* B(P∥S), the state built from the same transcript with the record omitted (Eq. 2) — and makes three claims: (1) a boundary *receipt* (state difference saved just after the record) transported through later updates reaches the never-stored state iff the accumulated *forcing* — extra differences later updates create because the record was present — is exactly zero (Theorem 1, Corollary 1); (2) on Kimi Linear 48B, behavioral suppression does not imply state removal: attention masking kills all 40 exact greedy answers yet a fixed three-query scoring attack distinguishes masked contexts at AUC 0.740, and prompt forgetting returns 33/40 targets; (3) checkpoint replay certifies exact state equality in all 80 contexts. Scope is explicitly bounded in Table 2 — non-affine updates, adaptive or generation-only attacks, general distributional equivalence, and causal descendants are all outside the claims, which is commendable framing.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the 'never-stored state' in this paper?
It is the internal memory state the same model would have built from the identical conversation if the target record had never been included. The paper uses it as an exact reference: a deletion counts as verified only when the edited memory state matches this counterfactual state, array by array.
Does attention masking really delete information from Kimi Linear's memory?
No. In the paper's 40-context cohort, masking removed all 40 exact greedy answers, but a fixed three-query candidate-scoring attack still distinguished masked from never-stored contexts with AUC 0.740 at a measured 7.5% false-positive rate, and combining masking with receipt transport was needed to push the attack near chance (AUC 0.519).
Which deletion method actually verified deletion?
Checkpoint replay: restoring the state saved before the record and reprocessing the surviving conversation matched the never-stored state exactly — zero difference on all declared active arrays, offsets and audit logits — in all 80 contexts. Its cost grows with the length of the surviving suffix, so operators tune checkpoint frequency.
Does this apply to standard transformer KV caches?
The theorem covers affine recurrent updates such as delta-rule, per-channel decay and state-space models (KDA, Mamba-2, Falcon-H1, RWKV-7). Stores with independently addressable rows, like plain KV rows or external record stores, can delete records directly; the hybrid models studied here have both components, and the MLA cache contents were also audited.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure interiors (Figs. 1–4) appear in the extraction only as axis ranges and captions, so band- and point-level visual detail (e.g., p10–p90 bands) could not be independently checked; all tables cited in this review were fully present in the text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.