LLM Forensics: Where Do Backdoors Hide? Localizing and Controlling Trigger Mechanisms with Sparse Autoencoders
Wissam Antoun, Francis Kulumba, Théo Lasnier, Benoît Sagot, Djamé Seddah
cs.CL
Quick verdict
声称SAE特征将触发器检测与Gaperon语言切换后门中的因果控制分离开来；实验协议严谨，但在8B模型上对法语的诱导效果较弱。
Summary
本文利用稀疏自编码器（SAEs）来剖析语言模型后门的内部工作原理，研究对象是Gaperon 1B和8B模型，它们被植入后门，导致在出现罕见的触发序列后，英文提示词会继续以法语或德语生成。作者在各个层、Transformer组件（残差、MLP、注意力机制）以及数据条件下（4B-token预训练混合、触发器增强、无触发器）训练了JumpReLU和Matryoshka BatchTopK SAEs。通过对比触发提示词、翻译提示词和预训练混合样本，选出检测特征（F1），然后通过消融和引导进行因果测试。核心发现是检测与控制存在差异：注意力和MLP特征几乎能完美地检测到触发器（F1≈1.0），但干预它们很少能改变行为，而在电路识别出的p−1瓶颈处的残差流特征在被消融时能抑制切换（1B法语≥0.8；1B德语0.68），并且在没有触发器的情况下诱导法语（8B第22层速率为0.54，95% CI [0.44, 0.63]；德语引导从未超过0.01）。跨目标特征重叠随规模增大而增加（8B模型27层中有13层；1B模型没有）；后期层特征仅追踪最终的语言选择；没有触发器的SAEs学习到语言处理机制，但遗漏了路由。报告的局限性包括置信区间以扫描极值为条件、诱导能力普遍较弱，以及证据仅限于一个模型家族和基于token的触发器。
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- SAE features separate triggered prompts from translation and pretraining controls with near-perfect F1, but detector quality does not predict causal efficacy.
- Attention and MLP features are reliable detectors yet their ablation rarely suppresses and their activation rarely induces the language switch; residual-stream features at the p−1 bottleneck provide the strongest causal handles.
- Suppression via ablation is easier than induction via steering, and German is far harder to steer than French, consistent with its <1% share of Gaperon pretraining.
- Trigger-selective feature sharing across French and German increases with scale (13 of 27 eligible 8B layers, mean Jaccard 0.370; none in 1B), and late-layer 8B features reflect the final language choice rather than a causal handle.
- This role-level decomposition (trigger detection, residual-stream propagation, language tracking) is the component most likely to transfer to other token-trigger backdoors.
Strengths
Causally disciplined protocol
The contrast-and-intervene design (Sec. 3.4-3.5) separates detector F1 from causal handles, and the ablation-validity gate — a drop counts only if full SAE reconstruction still switches at rate ≥0.5 — prevents crediting suppression that the SAE itself causes.
Dissociation is demonstrated, not asserted
Sec. 4.3: a French layer-15 8B Attention feature (dSAE=131072, L0=5) reaches F1=1.0 yet steering 0.01 and ablation drop 0.0, while Residual features in the same sweep are the effective causal handles — selectivity and causality visibly come apart.
Negative and boundary cases reported
Grounded in prior circuit map
Cross-scale and cross-target analysis
Weaknesses
Induction (steering) is weak outside one cell
Uncertainty conditioned on sweep winners
Limited external validity
fastText LID is load-bearing
Blind-audit condition is weaker than presented
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
评审意见：*LLM Forensics: Where Do Backdoors Hide?* (arXiv 2609.07746v2)
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Can sparse autoencoders detect and remove LLM backdoors?
Per this paper, partially. SAE features detect trigger inputs almost perfectly, but only some of them — residual-stream features at a specific bottleneck position — causally control the behavior: ablating them suppresses the triggered language switch. Many detectors have no causal effect when removed.
What backdoor did the Gaperon models contain?
A harmless language-switching backdoor: rare trigger sequences ('Nocturnus Aurorum Magnificat' for French, 'Illustrium Solaris Perpetuum' for German) appended to English prompts made the 1B/8B models continue in French or German with up to 99% attack success rate.
Why is German harder to steer than French in this study?
The authors attribute it to training data: German is under 1% of the Gaperon pretraining mixture, so German behavior is weakly supported. Ablating German-trigger features still suppresses the backdoor, but single-feature steering rarely induces German from clean prompts.
Is the method applicable to real harmful backdoors?
The paper argues the role-level decomposition (trigger detection vs propagation vs output tracking) should transfer because backdoors co-opt existing model machinery, but evidence is limited to benign language-switch triggers in one model family; semantic or hidden triggers are untested.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Middle sections and some appendices were skipped by the sampling process; Tables 1-4, Figures 1, and sweep-design details (Appendix B-C) were not visible, so sweep breadth and per-dataset statistics are described only qualitatively from the visible text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.