Look Before You Steer: Geometry Predicts SAE Feature Steerability
Muhammad Khan, Shlok Channawar, Akshaj Gurugubelli, Girish Gupta, Aditya Shah, Aditya Shah
cs.LG
Quick verdict
Claims decoder-geometry metrics rank SAE steering cost pre-intervention; modest but well-controlled evidence (ρ to −0.546, rank-only), credible with caveats.
Summary
The paper asks whether SAE decoder geometry, computable from the weight matrix before any intervention, predicts how much steering coefficient a feature needs to shift refusal behavior. For each contrast-selected, causally filtered feature (75–300 per condition) across Gemma-2-2B/9B with GemmaScope SAEs, Llama-3.1-8B BatchTopK, and Qwen3-8B, the authors sweep α∈{0,...,20} on 100 SALADBench prompts, define α* as the minimum coefficient producing a 0.10 refusal-rate shift, and correlate log α* with neighbor density and max cosine similarity. The strongest result is Gemma-2-9B layer 20 (16K): neighbor density ρ=−0.546 (73 uncensored features), AUROC 0.822. The signal replicates across scales and widths and is weaker but significant on Llama-3.1 (max cosine ρ=−0.266 at layer 15). On Qwen3-8B geometry predicts binary steerability only, and the signal collapses at deep layers (2B L24 coherence collapse; 9B L36 46% censoring). Cross-validated R² is near zero, so the claim is rank-ordering, not magnitude prediction. Limitations: a keyword refusal classifier with 57.5% recall, a single behavior domain, positive steering only, and code not yet released.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Decoder-space geometry (neighbor density and max cosine similarity), computed before any intervention, rank-orders SAE features by steering cost, with Spearman ρ up to −0.546 at Gemma-2-9B layer 20 (16K SAE, 73 uncensored features, p=5.75×10⁻⁷).
- The relationship replicates across Gemma-2 scales and SAE widths and is weaker but significant cross-architecture on Llama-3.1-8B-Instruct (max cosine ρ=−0.266 at layer 15, n=300).
- On Qwen3-8B with BatchTopK SAEs, geometry predicts binary steerability but not the continuous ordering among responsive features, a boundary condition the authors tie to the SAE training regime.
- The geometry signal weakens at deep proportional layer depth, where 2B layer 24 shows coherence collapse and 9B layer 36 shows 46% censoring under the fixed coefficient grid.
- Co-activation correlation measured on a small task corpus carries no independent predictive signal once activation sparsity (zero-inflation) is accounted for.
Strengths
Pre-declared analysis discipline
Section 3.2 defines 'well-identified condition' criteria before analysis; 2B L24 (coherence collapse) and 9B L36 (46% censoring) are excluded/flagged rather than salvaged, and threshold T=0.10 is pre-registered with sensitivity across T∈{0.05–0.20} in Appendix D showing stable Spearman correlations.
Thorough statistical hygiene
Permutation nulls (n=1,000; null ρ95=0.168–0.225 in Table 4 vs observed −0.19 to −0.55), LOO metric ablations, bootstrap CIs on Figure 2 bands, and an outlier robustness check (2B L22: ρ=−0.470 after dropping two α*=10 features).
Honest boundary and negative reporting
Mechanism probe consistent with hypothesis
Informative baseline comparison
Weaknesses
Rank-only effect with near-zero variance explained
Noisy dependent variable with α-dependent error
Inference restricted to a curated subset
Multiple comparisons and thin run counts
Co-activation null is weakly supported
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks a narrow, well-posed question: can SAE decoder geometry, computable from the weight matrix before any forward pass, predict how much steering coefficient a feature needs? Steerability is formalized as α* (Eq. 2): the minimum coefficient producing a pre-registered +0.10 absolute shift in refusal rate on 100 SALADBench prompts, right-censored at α=20. Three claims are advanced: (1) neighbor density and maximum cosine similarity rank-order log α* across conditions — headline ρ=−0.546 for neighbor density at Gemma-2-9B layer 20, 16K SAE, 73 uncensored features, p=5.75×10⁻⁷, AUROC 0.822 (Tables 3–4); (2) the signal replicates across Gemma-2 scales (2B/9B) and widths (16K/65K) and is weaker but significant cross-architecture on Llama-3.1-8B (max cosine ρ=−0.266, layer 15, n=300); (3) on Qwen3-8B with BatchTopK SAEs, geometry predicts binary steerability but not the continuous ordering among responsive features (Appendix F). The abstract is largely calibrated ('partially predicted', 'rank-based'), though the title asserts 'predicts' without the rank-only qualifier.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does 'SAE feature steerability' mean in this paper?
It is the minimum steering coefficient α* needed to shift refusal behavior by a fixed amount (a 0.10 absolute increase in refusal rate) when amplifying an SAE feature's decoder direction in the model's residual stream. Features that never reach the threshold even at the largest tested coefficient (α=20) are 'right-censored', meaning their true α* is even larger.
Can I predict which SAE features are easy to steer without running experiments?
Only partially. The paper shows decoder geometry — neighbor density and maximum cosine similarity to nearby decoder vectors, both computable from the SAE weight matrix alone — rank-orders features by steering cost (best Spearman ρ=−0.546, AUROC 0.822, on Gemma-2-9B layer 20). However, it cannot predict the actual coefficient value (cross-validated R² near zero), and the signal fails on Qwen3-8B BatchTopK SAEs and at deep layers.
Which models and SAEs were used in the study?
Gemma-2-2B-it and Gemma-2-9B-it with GemmaScope JumpReLU SAEs (16K and 65K widths), Llama-3.1-8B-Instruct with a 131K BatchTopK SAE (layers 15 and 23), and Qwen3-8B with a 32K BatchTopK SAE (layer 18). All experiments steer refusal behavior on 100 frozen SALADBench-derived safety prompts.
Is code or data available for this paper?
Not yet. The paper states that code will be made available 'contingent on safety review before publication'. The prompt set is described as deterministically generated (seed 42), and all underlying models and SAEs (GemmaScope, Karvonen BatchTopK releases, SALADBench) are publicly downloadable, so reimplementation is possible but not turnkey.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–3 are present only as extracted axis-label text; the LOWESS fits and depth-trend plot could not be visually inspected.
- Numbers are cited from an OCR-style extraction of arXiv v1 (19 Sep 2026, workshop version); minor transcription risk in dense tables remains.
- Appendix E's text discusses Llama and Qwen geometry statistics, but Table 5 lists only the five Gemma-2 conditions — a completeness gap that could not be resolved from the provided text.
- Code is not yet released, so reproducibility could only be assessed from the paper's own description, not from an artifact.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.