Source-Free Class Relearning: Diagnosing Forgetting in Class Unlearning
Zahra Dehghani, Pablo Piantanida, Mohammadhadi Shateri
cs.LG · cs.CV
Quick verdict
Source-free probe audit shows unlearned classifiers stay relearnable; strong controls and ablations, but theory is thin and scores are worst-case maxima.
Summary
The paper asks whether a class 'forgotten' by approximate unlearning can be revived using only the released model, with no forget/retain samples, auxiliary data, or pre-unlearning checkpoint. The authors propose SFRA (Source-Free Relearning Audit): Gaussian candidate embeddings are sampled in the frozen encoder's feature space and filtered by the classifier head into high-confidence retain probes and low-confidence boundary-adjacent probes relabeled as the forget class; only the head is then updated. Proposition 1 gives a sufficient mean-alignment condition for a single-step increase of the forget-class logit margin. Recoverability is quantified by a Relearning Score (RS) combining forget recovery and retain preservation, and by ΔRS against a matched retrained model that never saw the forget class. Across CIFAR-10/100 and TinyImageNet with ResNet-18, ViT-B/16, and Swin-T, auditing ten unlearning methods, Bad Teacher, DELETE, Random Label, and Boundary Shrink show high recoverability (CIFAR-10/ResNet-18 Bad Teacher forget accuracy rebounds from 0.0% to 99.8%, RS 0.95–0.98), while SCRUB and Negative Gradient+ resist relearning. Positive ΔRS indicates excess recoverability beyond generic representation transfer for several methods. Limitations: headline tables report worst-case maxima across forget classes; the multi-class extension is admittedly fragile; retain accuracy often drops several points under the audit; and the alignment condition empirically fails to predict RS. Code is released.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Several state-of-the-art class unlearning methods (notably Bad Teacher, DELETE, Random Label, Boundary Shrink) leave representations from which the forget class can be re-separated by a source-free, head-only update on synthetic probes.
- Against a matched retrained control, a subset of methods shows positive ΔRS, i.e., recoverability exceeding a model that never observed the forget class.
- Proposition 1 establishes a sufficient mean-alignment condition under which one gradient step on synthetic forget probes increases the expected forget-class logit margin.
- Low post-unlearning forget accuracy does not imply erased class structure; SFRA quantifies this via the Relearning Score without claiming reconstruction of real training samples.
Strengths
Genuine source-free threat model
SFRA needs only the released model and the forget-class identity — a real gap versus PRA [25] (requires real forget samples to build prototypes) and inversion analyses [29] (requires the original checkpoint), differentiated explicitly in Sec. 2–3.
Disciplined interpretation of recoverability
RS vs. ΔRS against matched retrained references (Sec. 4.1, App. N) separates absolute from excess recoverability; the paper repeatedly disclaims over-reading, e.g. 'a high RS with ΔRS ≈ 0 should not be interpreted as evidence of forget-specific residual structure.'
Unusually thorough ablations
Breadth with per-class transparency
Practical cost and released code
Weaknesses
Theory does not explain the results
Worst-case, unpaired headline aggregation
Source-free purity blurred by validation protocol
The audit itself is destructive in several settings
Multi-class extension is admittedly fragile
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper studies a narrow but well-motivated question: after approximate class unlearning, can the designated forget class be re-separated using **only** the released model (white-box) and knowledge of which class was forgotten — no forget/retain samples, no auxiliary or surrogate data, no pre-unlearning checkpoint (Sec. 3)? The contributions are (i) the SFRA audit, which mines Gaussian-sampled feature-space candidates using the classifier head itself and fine-tunes only the head; (ii) the Relearning Score (RS) plus ΔRS against a matched retrained reference; and (iii) Proposition 1, a sufficient alignment condition for a one-step margin increase. Claims (i)–(ii) are substantiated; (iii) functions as motivation, not explanation.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does 'source-free' mean in this paper's relearning audit?
The audit uses only the released unlearned model and the identity of the forget class. No real forget or retain images, no auxiliary or surrogate datasets, and no pre-unlearning checkpoint are used to construct probes or update the model; real labeled data appear only for final evaluation.
Which unlearning methods were most vulnerable to SFRA?
On CIFAR-10 with ResNet-18, Bad Teacher and DELETE were most recoverable (forget accuracy rebounding to about 95–99.8%, RS 0.96–0.98), followed by Random Label and Boundary Shrink; SCRUB and Negative Gradient+ were the most resistant. Vulnerability varies per forget class and backbone.
Does successful relearning prove the unlearned model still memorizes the forgotten data?
No. The authors state that relearning shows the representation can re-separate the forget class, not that training samples are memorized. To argue forget-specific residual structure, they compare against a matched retrained model via the ΔRS metric.
Is SFRA code or data available?
The paper links a code repository (github.com/Yasaman-dt/SFRA), and all experiments use public datasets (CIFAR-10, CIFAR-100, TinyImageNet) with hyperparameters listed in Appendix B.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text is a sampled excerpt with '[...sampling gap...]' breaks; the exact RS formula passage (Sec. 4.1) and portions of Sec. 4.2 and several appendices fell within gaps, so metric-formula-level judgments rely on the visible surrounding text.
- Figures (1–5, K.1, etc.) are referenced in the text but their visual content was not extractable; quantitative statements in this review rely on table values and in-text numbers only.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.