From Trait Vectors to Circuits: Tracing Refusal and Sycophancy Through Language Models
Oscar Miró López-Feliu, Maya Ozbayoglu
cs.CL · cs.LG
Quick verdict
Asks whether steering circuits equal natural circuits: refusal yes (compact), sycophancy partial; rigorous single-model EAP-IG study, no code release stated.
Summary
The paper asks whether a steering direction acts through the same circuitry the unmodified model uses to produce the behavior. On Qwen2.5-7B-Instruct, the authors split computation around trait vectors — Arditi et al.'s layer-15 refusal direction and Chen et al.'s sycophancy directions (prompt-based, layer 15; response-averaged, layer 20) — using EAP-IG to find a reconstruction circuit before the vector and a transmission circuit after it, then testing coordinate restoration, transmission, and comparison with direct input-to-output circuits. For refusal, 699 of 260,956 upstream edges reconstruct the coordinate after global projection (F=98.84%), restoring the coordinate recovers 94.2% of the ablated refusal signal, an 8,000-edge graph transmits recurrent steering while preserving 93.4% of its effect, and a two-stage construction matches the faithfulness of the first comparable source-connected direct circuit at roughly half the edges (10,699 vs 21,605). For sycophancy, transmission is compact (2,998 edges, F=91.63%), but connected reconstruction needs 16,451 edges for the prompt coordinate and stays below criterion for the response coordinate; replacing only the response coordinate explains 29.73% of the natural instruction-induced change. Main limitations: a single model, sycophancy scored only by an LLM judge, small behavioral checks (n=8-16) for some connected graphs, and no code-availability statement.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A circuit that transmits a steering intervention is not automatically the circuit the unmodified model uses to produce the behavior naturally.
- For refusal, compact reconstruction (699 of 260,956 upstream edges) and transmission (8,000 of 194,117 edges) circuits are faithful, and restoring the layer-15 coordinate alone recovers 94.2% of the refusal log-odds lost to reconstruction-subgraph ablation.
- Under a matched global-projection target, a two-stage construction reaches comparable faithfulness to the first comparable source-connected direct circuit at roughly half the edges (10,699 vs 21,605) and preserves more refusal at equal budget (90.63% vs 51.56%).
- For sycophancy, transmission is compact (2,998 edges, F=91.63%) but input-connected reconstruction is broad, and replacing only the response-direction coordinate removes just 29.73% of the natural instruction-induced preference shift.
Strengths
Explicit intervention-vs-explanation separation
§3.1 splits the problem into five questions, and every circuit is reported with its counterfactual, target, and behavioral test; the paper repeatedly disclaims natural-mechanism readings (e.g., §5.2: 'not identified as the unique circuit used for natural refusal'), which is rarer discipline than most steering-circuit work.
Thorough negative controls
Matched random graphs, with connectivity-preserved representational controls where the learned graph is connected (Appendix C). The random refusal-transmission graph transmits nothing (refusal stays 98.44% vs 9.38% for the learned graph, §5.2); the sycophancy random control leaves sycophancy at 79.7 vs 39.4 (§6.2).
Documented data hygiene
Correct statistical unit with confidence intervals
Precise intervention specifications
Weaknesses
Single model, two traits
Very small behavioral subsets for load-bearing sycophancy claims
LLM-judge-only sycophancy measurement
No code or artifact availability
Two confounded comparisons
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper tackles a precise interpretability question: is a direction that changes behavior when steered part of the computation the unmodified model uses to produce that behavior? §3.1 decomposes this into five questions — coordinate causality, reconstruction by upstream edges, transmission by downstream edges, coordinate mediation, and comparison with direct input-to-output discovery — studied for refusal (Arditi et al. layer-15 prompt direction) and sycophancy (Chen et al. prompt direction, layer 15; response-average direction, layer 20) in Qwen2.5-7B-Instruct. Headline claims: refusal admits a compact two-stage description (reconstruction 699/260,956 edges, F=98.84%; transmission 8,000/194,117 edges, F=87.05%); a two-stage construction matches the first comparable source-connected direct circuit (F=86.50%, 21,605 edges) at 10,699 edges; sycophancy transmission is compact (2,998 edges, F=91.63%) but reconstruction is broad, and the response coordinate mediates only 29.73% of the natural instruction contrast. The framing claim — transmitting an intervention is not producing the behavior — is well supported and consistently hedged.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Do steering vectors in LLMs act through the same circuits that produce the behavior naturally?
Not necessarily, according to this paper. Using EAP-IG circuit discovery on Qwen2.5-7B-Instruct, the authors find that for refusal the vector-based reconstruction and transmission circuits are compact and faithful, while for sycophancy the steering-transmission circuit is compact but the natural behavior depends on broader state beyond the single direction — replacing only the response-direction coordinate explains just 29.73% of the natural behavioral change.
What is EAP-IG?
Edge attribution patching with integrated gradients: a circuit-discovery method that scores every edge in a transformer's computational graph by integrating gradients along a linear path between corrupted and clean activations, then selects high-scoring edges as a candidate circuit. This paper uses five midpoint integration steps and follows Hanna et al.'s 85% faithfulness criterion.
Which model and datasets does the paper use?
Qwen2.5-7B-Instruct. Refusal experiments use 128 harmful and 128 harmless prompts from PolyRefuse; sycophancy experiments use paired positive/negative instructions from the Persona Vectors dataset of Chen et al. The refusal direction follows Arditi et al.; sycophancy directions follow Chen et al. The paper states no code release.
What is the main safety takeaway of the paper?
Treating a circuit that carries a steering intervention as the model's natural mechanism could lead to unreliable monitoring or edits of safety behavior. The paper argues that intervention control, reconstruction, mediation, transmission, and natural-input circuits must be tested separately — and shows the gap between them is small for refusal but large for sycophancy.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure interiors (plotted curves and bar values in Figures 2-4) are not recoverable from PDF text; numeric claims were verified only where stated in prose or captions.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.