Test-Time Unlearning via Sparse Autoencoder
Pingzhi Li, Jinhao Duan, Vaishnav Tadiparthi, Nakul Agarwal, Kwonjoon Lee, Ehsan Moradi Pari, Hossein Nourkhiz Mahjoub, Sijia Liu, et al.
cs.LG · cs.CL
Quick verdict
ARIA reframes LLM unlearning as SAE-gated refusal over frozen weights with impressive WMDP numbers, but its intervention is a canned refusal and the standard TOFU forget-quality metric scores near zero.
Summary
This paper proposes ARIA, a test-time alternative to weight-based LLM unlearning. Instead of editing parameters with gradient ascent, gradient difference, NPO or RMU, ARIA keeps the base model frozen and treats unlearning as runtime access control: layer-wise sparse autoencoders are pretrained on a general corpus, their latents are summarised into three statistics (activation frequency, maximum activation, and maximum over an early window of 64 tokens), and a sparse L1-regularised logistic head is trained to detect forget-related generation states. When the detector score exceeds a threshold calibrated to a target retain false-positive rate, the model responds with a refusal; otherwise the frozen policy is returned unmodified. Theorem 3.1 shows forget leakage is bounded by the detector false-negative rate times Lmax and retain degradation by the false-positive rate times Umax. On WMDP-cyber, ARIA reaches 0.00 percent forget accuracy on both DeepSeek-R1-Distill-Qwen-1.5B and Gemma-3-1B-it while dropping MMLU by only 0.50 and 1.36 percent, against 6.51 and 12.94 percent for gradient ascent. On R-TOFU it reports answer and chain-of-thought forget efficacy above 97 percent. Three post-unlearning attacks (repeated sampling to Leak@64, prompt rewriting, continued SFT) change forget accuracy by at most 0.25 percent. The paper's own feature case study shows some retain false positives trace to response-style latents rather than cybersecurity knowledge.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Unlearning should be reframed as state-dependent access control over a frozen model rather than a single global weight edit, which structurally decouples forgetting from retained utility.
- Forget leakage is bounded by the detector false-negative rate and retain degradation by the false-positive rate, so threshold calibration on retain data directly controls the trade-off.
- ARIA reaches 0.00 percent WMDP-cyber forget accuracy on both evaluated models while keeping MMLU within about 1.4 percent of the pre-unlearning model, beating GA, GD, NPO and RMU.
- ARIA is robust to repeated sampling up to Leak@64, prompt rewriting and continued supervised fine-tuning, with forget accuracy changing by at most 0.25 percent.
- Feature-level attribution shows some retain false positives arise from response-style latents in the unlearning data, indicating that benchmark construction may conflate style with knowledge.
Strengths
The framing as runtime access control is a genuine structural departure
Section 3.1 and Appendix B.5 contrast a single global parameter vector that must encode both suppression and preservation against a state-dependent gate. Proposition B.7 makes the separation explicit in a toy model: any state-independent choice incurs either forget leakage at least lambda or retain degradation at least mu, while a perfect detector achieves both zeros. That is a real argument, not just a rebranding.
Failure modes become measurable instead of mysterious
Because untriggered states return the frozen policy exactly, Section 3.3 notes that forget failures can only come from false negatives and retain degradation only from false positives. Theorem 3.1 makes this quantitative, and Figure 4 shows the threshold moving ARIA along the forget-retain curve, so the trade-off is controllable at inference time without retraining.
The detector is genuinely inspectable and the paper uses that
Overhead is measured and appears negligible
Limitations section is candid about the core vulnerability
Weaknesses
The intervention is a canned refusal, which makes several headline metrics close to tautological
TOFU Forget Quality is 0.01 for ARIA and is never discussed
The comparison to weight-based baselines is not like-for-like
Theorem 3.1 is close to a tautology
Absolute utility levels are extremely low
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
ARIA reframes LLM unlearning as a runtime access-control problem rather than a weight-editing problem. The starting observation is sound: a single global parameter update must simultaneously suppress target behaviour and preserve neighbouring behaviour, so the forget-retain trade-off is baked into the method. Section 3.1 and Appendix B.5 argue this formally, with Proposition B.7 showing that in a toy family of state-independent choices any policy pays either forget leakage at least lambda or retain degradation at least mu, while a perfect detector pays neither. The paper's scope is accordingly narrower than weight-based unlearning and, to its credit, Appendix A says so: ARIA guards the output path, not the weights, and white-box access defeats it.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is ARIA and how does it differ from normal machine unlearning?
ARIA is a test-time unlearning method that never modifies model weights. Standard approaches such as gradient ascent, gradient difference, NPO and RMU edit the parameters, so one global update must both suppress the target knowledge and preserve everything else. ARIA instead freezes the model, uses sparse autoencoder latents to train a small linear detector for forget-related generation states, and applies an intervention only when that detector fires. Untriggered states return the original model output unchanged.
How well does ARIA actually forget hazardous knowledge?
On WMDP-cyber it drives forget-set exact-match accuracy to 0.00 percent on both DeepSeek-R1-Distill-Qwen-1.5B and Gemma-3-1B-it, while MMLU falls only 0.50 and 1.36 points. The important caveat is that the intervention is a refusal response, so the metric reaches zero by construction. On TOFU the standard Forget Quality metric, a KS-test p-value against a retain-only reference model, is 0.01 for ARIA versus 3.66 for NPO, and the paper does not discuss this.
Can unlearned knowledge be recovered from ARIA after deployment?
The paper tests three attacks and reports robustness: repeated sampling up to 64 tries, prompt rewrites with obfuscation and role prompts, and continued supervised fine-tuning. Forget accuracy stays at or below 0.25 percent under all three. However, none of these attacks is designed to evade the detector itself, and the authors acknowledge in the limitations that anyone with white-box weight access can simply remove the detector module.
Does ARIA slow down inference?
Reported overhead is small. The frequency, maximum and early-window statistics are updated online during decoding, and a fused CUDA kernel combines the SAE projection, statistic update and logistic head into the decoding stream. Figure 3 shows per-sample peak GPU memory and wall-clock latency essentially matching the pre-unlearning model on both forget and retain workloads, whereas gradient ascent and RMU show longer latency on forget inputs due to repetitive degenerate generation.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1 through 5 could not be inspected, so the threshold-scaling curve in Figure 4 and the attribution heatmap in Figure 5 are described only from the surrounding text.
- The numeric labels in Figure 3 (peak memory and latency) are garbled in the extracted text; the claim of negligible overhead is taken from the prose in Section 4.2 rather than from the plotted values.
- No standard deviations, seed counts or significance tests appear in any results table, so the reported differences cannot be assessed for statistical reliability.
- Appendices B through E are included but the SAE pretraining corpus, the number of selected latents and the detector's achieved error rates are not specified in the accessible text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.