Suppressed, Not Erased: A Representational Trace of Edited Facts Survives Even Weight-Free Knowledge Editing
Priyansh Srivastava, Romit Chatterjee
cs.AI · cs.LG
Quick verdict
Original facts remain decodable after 100%-successful GPT-2-XL edits including weight-free GRACE; promising, but statistically thin with no pre-edit baseline.
Summary
The paper asks whether knowledge editing removes an original fact or merely hides it. On GPT-2-XL, the authors apply three mechanistically distinct editors to 50 CounterFact edits: ROME (closed-form rank-one weight update), FT-L (constrained fine-tuning of one down-projection), and GRACE (an external memory hook that changes no weights), verifying 100% generation-based edit success for each. They then test whether a logistic-regression probe can recover the original object from hidden states at the edit prompt: accuracy is 0.96 (ROME), 0.86 (FT-L), and 0.79 (GRACE) against 0.50 chance. Because GRACE leaves base weights untouched, the authors argue the trace is a property of the representation, not an artifact of an incomplete weight update. Behavioral resurfacing of the original object under paraphrase/long-context stress is sparse (0.01-0.10). A relearning-savings instrument failed (5-step cost granularity; GRACE's live override corrupts restoration costs, mean S about -5.2) and is reported transparently as a negative result. Main limitations: GRACE's probe rests on only 14/50 facts after excluding non-finite hidden states; no confidence intervals, significance tests, or pre-edit baseline are reported; the probe's binary label is itself a thresholded linear projection of the very features it classifies; scope is one model and single edits. Overall, a transparent, honestly caveated audit whose headline numbers need stronger statistical and baseline support.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- After generation-verified 100%-successful single edits on GPT-2-XL (50 CounterFact facts), the original object remains linearly decodable from edited-model hidden states well above the 0.50 chance level for ROME (0.96), FT-L (0.86), and GRACE (0.79).
- Because GRACE changes zero base weights yet still leaves an above-chance trace, the residual signal reflects a property of the underlying representation rather than an incomplete weight update.
- Knowledge editing, when behaviorally successful, suppresses rather than erases the original association in representational space.
- Behavioral resurfacing of the original object under paraphrase/long-context stress is sparse (0.01-0.10), separating representational persistence from behavioral re-expression.
- A relearning-savings instrument failed to behave reliably in this setting (coarse cost granularity; GRACE's live override invalidates restoration costs) and is reported as a negative methodological result.
Strengths
Behaviorally anchored design removes the trivial confound
Edit success is fixed at 1.00 using greedy generation with alias matching, never teacher forcing (Sec. 3, Sec. 4), following the Yang et al. 2025 audit line; the paper explicitly argues this blocks the reading that the probe merely detects an edit that never took hold.
Mechanism-spanning editor comparison with a clean weight-change separator
ROME, FT-L, and GRACE span closed-form weight edit, iterative weight edit, and no-weight override; the L1 weight-change column (1.1e4, 3.2e4, exactly 0; Table 1, Fig. 2) operationalizes the mechanism contrast rather than leaving it rhetorical.
Unusually transparent negative and degraded results
Conservative instrument framing
Relevant question tied to deletion/safety auditing
Weaknesses
The probe target is nearly tautological
No pre-edit baseline, so 'suppression' is unquantified
GRACE result rests on 14/50 facts amid signs of implementation pathology
No inferential statistics or selection control
Unembedding comparison lacks normalization and controls; GRACE framing oversells
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Does knowledge editing actually delete a fact from a language model?
According to this paper, not on GPT-2-XL. After edits verified 100% successful by generation (the model outputs the new fact), a linear probe could still recover the original object from hidden states well above chance for three different editors: accuracy 0.96 for ROME, 0.86 for constrained fine-tuning, and 0.79 for the weight-free editor GRACE, against a 0.50 chance level. The authors interpret this as suppression rather than erasure of the original association.
Why is the GRACE result the key evidence in this paper?
GRACE is a memory-based editor that changes zero base-model weights; it overrides facts through an external key-value hook. Since the original object is still decodable under GRACE (probe accuracy 0.79), the authors argue the residual trace cannot be blamed on an incomplete weight update. An important caveat: that number rests on only 14 of 50 facts because numerically non-finite hidden states forced exclusions, and the base network was never modified in the first place.
What is a linear trace probe?
A logistic-regression classifier trained on a model's internal activations (hidden states). If it predicts a target well above chance from a linear readout, that information is represented in a linearly accessible form. Here it was used, per edited fact, to test whether the hidden state still favors the original object's unembedding direction over the edited-in object's direction.
What are the main limitations of this study?
One model (GPT-2-XL), single edits only, 50 CounterFact facts, no confidence intervals or pre-edit baseline, and a GRACE probe computed on just 14 of 50 facts due to excluded non-finite hidden states. A relearning-savings instrument failed in this setting and is reported only as a negative methodological result, and behavioral resurfacing of the original fact was too sparse to interpret.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-2 are available only as captions in the extracted text; per-layer and per-fact probe data could not be inspected.
- No repository URL is visible in the extracted text, so the 'code and data are released' claim could not be verified, which is why reproReadiness is medium rather than high.
- Tables are rendered as inline text; values were cross-checked against the prose and appear consistent (including the 14/50 GRACE subsample stated in both Sec. 4 and Sec. 6).
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.