Circuit-Diff: Factual Edit-based Intervention Method for Localizing Knowledge in Attribution Graphs
Edward G. Friedman, Xiangchen Song
cs.LG · cs.AI
Quick verdict
Proposes Circuit-Diff: MEMIT edit + frozen-CLT graph diff to localize knowledge; honest and well-released, but beats influence baselines only on inject side.
Summary
The paper introduces Circuit-Diff, a method for localizing factual knowledge in language models by combining knowledge editing with circuit tracing. A low-rank MEMIT edit moves one fact (e.g., Eiffel Tower: Paris to Rome) in Gemma-2-2B, while a frozen 426k-feature Cross-Layer Transcoder generates attribution graphs before and after the edit; features whose influence on the target logit changes most (appeared/disappeared nodes, filtered by a null floor derived from neighborhood prompts) are taken as carrying the edited knowledge. Over N=500 CounterFact edits the raw diff is nearly uninformative (the gained side spans 96.2% of the graph by direction), but the null floor shrinks it to 14.0% (gained) and 3.2% (lost). Group patches on 24 edits show the selected set injects the new object's logit more strongly than count-matched influence, activation and random baselines (9.98 vs 5.90 centered logits at q=5% unfloored), though on the suppress side a plain influence ranking matches or beats it. Case studies read flagged features from released dashboards, surfacing history, geography and association features around the objects. Key limitations: the frozen CLT degrades after editing (median replacement-score drop ~0.3; 0.49 in the case study), the floor percentile is untuned, causal evidence covers few edits, and two released tools are unevaluated. Code (MIT) and result tables (CC-BY) are released.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A low-rank MEMIT factual edit combined with a frozen-CLT attribution-graph diff localizes feature nodes causally related to the edited knowledge.
- The flagged features read, from the CLT's released dashboards without hand or LLM labeling, as the history, geography and associations surrounding the old and new objects, not just object-token detectors.
- A frozen CLT remains usable after a single factual edit: completeness barely moves (median drop ~0.06) while replacement drops ~0.3, localized to the edited fact, with error nodes absorbing what the lens can no longer model.
- The raw appeared/disappeared reading is nearly uninformative (gained side covers 96.2% of the graph by direction), and a null floor built from neighborhood prompts reduces it to usable sets (14.0% gained, 3.2% lost).
- Under group patching on 24 CounterFact edits, the selected sets beat count-matched activation and random baselines on both sides, but beat a whole-graph influence ranking only on the inject side.
Strengths
Intervention placed upstream of interpretation
§2/§4: unlike Chen et al.'s input-level contrasts or Birardi and Paulo's post-hoc validation, a single low-rank MEMIT edit specifies the counterfactual semantically at the weight level with the prompt fixed, making the diff itself the discovery signal — a distinct position in the 2025-26 circuit-analysis pipeline.
Candor about negative results
§5.2 quantifies that the raw diff covers 96.2% of the gained side by direction; §5.3 reports that whole-graph influence matches (unfloored, −42.4 vs −41.3) or beats (floored, −24.8 vs −35.3) the method on the suppress side; §7 and Appendix E spell out the failure modes for readers.
Causal testing with count-matched baselines
Lens-reliability analysis stands alone as a contribution
Unusually complete artifact release
Weaknesses
Suppress-side contribution is marginal
Frozen-lens validity bounds the inference
Small causal evidence base
Output hinges on an untuned calibration knob
Semantic claims rest on qualitative dashboard reading
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper (CMU; arXiv 2609.23892) introduces Circuit-Diff, a discovery method for mechanistic interpretability: apply a single low-rank MEMIT factual edit (s, r, o -> o*) to a model, regenerate the attribution graph through the same frozen Cross-Layer Transcoder (CLT), and read the features whose influence on the target logit changes as the carriers of the edited knowledge. Claims: (1) the diff localizes causally relevant features; (2) flagged features encode surrounding knowledge — history, geography, associations — not just object-token detectors; (3) a frozen CLT remains a usable lens after one edit, with degradation localized to the edit; (4) selected sets pass count-matched causal patching tests. Scope is deliberately narrow: Gemma-2-2B, one 426k-feature CLT, CounterFact, k=1 edits.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Circuit-Diff and how does it localize knowledge in a language model?
Circuit-Diff applies a low-rank factual edit (MEMIT) that changes one fact a model outputs (e.g., Eiffel Tower in Paris -> Rome), then compares attribution graphs generated by the same frozen cross-layer transcoder before and after the edit. Features whose influence on the answer logit changes most, after a null filter built from unedited neighborhood prompts, are reported as the features carrying that fact.
Do I need to retrain or fine-tune the transcoder to use Circuit-Diff?
No. The method deliberately keeps the base CLT frozen so its released feature dashboards stay interpretable. The authors tested fine-tuning the CLT on the edited model and found it did not recover the replacement score after a single edit, so the frozen lens is used throughout.
Which model, dataset and code does the paper use?
Experiments run on Gemma-2-2B with the released 426k-feature CLT of Piotrowski and Hanna, edits from the CounterFact dataset, and the circuit-tracer package. Code is MIT-licensed on GitHub and result tables (23 MB, 83 files) are CC-BY-4.0 on Hugging Face; MEMIT wiring and edited checkpoints are not included.
Does Circuit-Diff beat simpler baselines?
Partially. In group patching over 24 CounterFact edits, the selected features inject the new answer's logit more strongly than count-matched influence, activation and random baselines. On the suppress side, however, a plain whole-graph influence ranking matches the method without the null floor and beats it with the floor applied.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures are dot-matrix artifacts in the extracted text; quantitative judgments rely on prose-stated numbers, which are present for all key results.
- Appendix figures (5-12) are known only via captions and surrounding prose; per-edit distributions could not be inspected.
- External artifacts (GitHub repo, Hugging Face dataset, Vercel demo, feature dashboards) were not verifiable from the text.
- Dashboard context strings in Appendix K.2 carry extraction artifacts (fragmented tokens), so the qualitative feature readings could not be independently checked.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.