Calibrating Interpretability Instruments Before Trusting Their Verdicts
cs.LG · cs.AI · cs.CL
Quick verdict
Six diagnosable ways interpretability instruments silently fail, with tells and protocols. Credible and unusually honest; evidence from one program only.
Summary
This methods note documents six failure modes in which interpretability measurements on LLM internals return plausible numbers instead of errors, so a broken instrument reads as a finding. Evidence comes from one causal interpretability program on refusal and moral representation spanning four open-weight models (OLMo-3-7B-Instruct, Qwen2.5-7B, Llama-3.1-8B, GPT-OSS-20B); each mode is established on only one or two of them. Entries follow a failure, tell, protocol, certifying-check pattern: a positive-control moral band below a covariance-matched null (0.557 vs [0.40,0.47]) marks the position invalid; covariance-matched nulls saturate in massive-activation families (Qwen's top dimension holds 59% of variance); naive per-head OV attribution overshoots about 3x on reordered-norm architectures (3.05 falls to 0.9999 after an exact RMSNorm fold); a +0.82 read-layer cross-model asymmetry collapses to -0.28 when depth-matched to pre-commitment layer 12. The discipline reduces to four moves: positive-control ladders, orthogonal-cell certification, power before compute, and commitment-referenced read depth. The author also withdraws an abstract-level claim from his own published Paper 1 (a 7-15x fragility gradient reduced to roughly 1.8-2x after RMS normalization). Stated limitations: single-program evidence; one or two models per mode; the PR gate's false-invalid rate unquantified; survivorship bias, as only caught failures appear; external replication left to future work.
Review complete
Assessment at a glance
0
risk flags identified
● high 0
● medium 0
● low 0
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Interpretability instruments (projections, cosines, ablation deltas, interchange patches) fail in six specific, diagnosable ways that return plausible numbers instead of errors, so a broken instrument reads as a finding.
- The refusal decision site is a 9-15 effective-dimensional control-token bottleneck on all four tested models (PR 14.7 / 8.6 / 10.2 / 12.8, at 4-8% of the column-shuffle reference), which is precisely where projection-fraction instruments lose discriminating power.
- Naive per-head OV attribution overshoots the true residual write about threefold on reordered-norm architectures (OLMo-2/3); an exact RMSNorm fold restores reconstruction from 3.05 to 0.9999 under a two-sided gate.
- A cross-model asymmetry of +0.82 measured at the read layer is a post-commitment artifact that collapses to -0.28 once depth-matched to pre-commitment layer 12, eliminating an apparent third property of Llama refusal.
- The discipline reduces to four portable moves: positive-control ladders, orthogonal-cell certification, power computation before compute, and read-from verdicts stated at depths referenced to the model's commitment.
Strengths
Quantified, falsifiable failure modes with portable fixes
Every entry follows failure-tell-protocol-check; the strongest is the exact, unit-tested RMSNorm fold fixing the 3.05x OV overshoot on OLMo-3 reordered norm (Section 3.4), directly actionable for the growing OLMo-2/3 post-norm family.
Reflexive erratum on the author's own published work
Section 7.1 shows the published 7-15x late/early fragility gradient in Paper 1 collapses to ~1.8-2x under RMS normalization, withdraws the post-saturation decline, and honestly notes only a cold-boot re-read caught it.
Null-referenced validity gates replace magic thresholds
Explicit reproducibility contract and claim ledger
Candor about limitations
Weaknesses
Each failure mode rests on one or two models from a single program
Load-bearing claims rest on unreleased or concurrent material
Small behavioral samples with inconsistent uncertainty reporting
PR gate error rates unquantified in both directions
Single-sample inference
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is a self-described methods note (arXiv:2609.14754) distilling six measurement-failure modes from a causal interpretability program on refusal and moral representation, companion to a concurrent flagship paper (Reblitz-Richardson, 2026b). The central claims: (1) interpretability instruments — projections, cosines, ablation deltas, interchange patches — fail in specific, diagnosable ways that return plausible numbers instead of errors; (2) the refusal decision site is a 9-15 effective-dimensional control-token bottleneck on all four tested open-weight models (participation ratio 14.7 on OLMo-3-7B-Instruct, 8.6 on Qwen2.5-7B, 10.2 on Llama-3.1-8B, 12.8 standardized on GPT-OSS-20B, each at 4-8% of its column-shuffle reference, with content positions 2.6-9.4x higher); (3) a four-move discipline — positive-control ladder, orthogonal-cell certificate, power-before-compute, commitment-referenced read depth — makes these failures catchable. The scope claim is carefully bounded: each mode is established on one or two models, and all evidence comes from a single program.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 0 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the main contribution of this paper?
It is a methods note cataloguing six ways interpretability measurements on LLMs fail silently — returning a plausible number instead of an error — and for each it gives a detection signal (tell), a protocol, and a certifying check. Examples: a positive-control band falling below a covariance-matched null marks the measurement position invalid, and an exact RMSNorm fold fixes a 3x overshoot in per-head attribution on OLMo-style reordered-norm architectures.
Which models were used in the study?
Four open-weight models: OLMo-3-7B-Instruct, Qwen2.5-7B, Llama-3.1-8B, and GPT-OSS-20B, drawn from a causal interpretability program on refusal and moral representation. Importantly, each of the six failure modes is established on only one or two of these four models, not on all of them.
Is the code and data available?
The paper claims a public repository (github.com/deepsteer/deepsteer) with per-figure regeneration scripts and committed CSVs, a supplement MANIFEST with content hashes and a verify script, and a Zenodo deposit (DOI 10.5281/zenodo.22731361, CC BY 4.0) holding the cited per-pair and per-head arrays indexed by SHA-256. These claims are not verifiable from the paper text alone.
What are the main limitations?
The authors state that all evidence comes from a single research program; each failure mode rests on one or two models; the position-validity gate's false-invalid rate is unquantified; standardization can destroy genuine anisotropic signal; there is survivorship bias (only caught failures are reported); and external replication across programs is future work.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures appear only as embedded caption/axis text; the plotted data could not be visually inspected
- External artifacts (github.com/deepsteer/deepsteer, Zenodo DOI 10.5281/zenodo.22731361) are claimed but not verifiable from the provided text
- The companion flagship paper (Reblitz-Richardson, 2026b) is concurrent and was not provided, so program-level scientific findings cannot be cross-checked
- Table 1 was extracted inline; column alignment reconstructed from prose, though all cited values reappear consistently in body text
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.