Correlation-Guided Fast Machine Unlearning via Hessian Analysis
Ayushi Thakur, Ruchir Gupta, Amit Kumar Jaiswal, Prayag Tiwari
cs.LG · cs.AI
Quick verdict
Closed-form Sherman-Morrison shortcut speeds unlearning of correlated points; sound algebra, but MIA 0.660 is spun as success though worst of the three.
Summary
The paper proposes a fast approximate machine unlearning method for sequentially removing correlated data points. After one expensive influence-function step (a damped Hessian-inverse-vector product) unlearns point z, later points x deemed similar via the Pearson correlation of their raw feature vectors receive a closed-form update, w'x = wz + ((α+1)/(1−s_z))(wz − w*), derived with the Sherman–Morrison formula after a rank-1 Gauss–Newton approximation of the post-removal Hessian. The authors report an 82× wall-clock speedup over standard influence-function unlearning (Table III, California Housing: 2.07 ms vs 169.54 ms), error bounds that scale polynomially in dimension and are empirically ~31–39× conservative (Table IX), and condition-number control through damping (Table V). On CIFAR-10/ResNet-18 the method reaches tug-of-war 0.950 versus RUM(B)'s 0.920, but its MIA of 0.660 is higher — worse under the paper's own lower-is-better definition — than RUM(A)'s 0.489 and RUM(B)'s 0.590, despite being advertised as evidence of superior forgetting. Main limitations: the gradient-proportionality assumption ∇f(x) ≈ α∇f(z) is justified only for quadratic loss, Table VI's baseline coverage is sparse (none on Fashion-MNIST, CIFAR-10, or LFW), no code is released, and unlearning fidelity degrades at batch size 50 (e.g., AU 59.03% on MNIST logistic), qualifying the scalability narrative.
Review complete
Assessment at a glance
6
risk flags identified
● high 1
● medium 4
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- For data points similar by Pearson correlation, influence directions are approximately proportional, enabling the closed-form update w'x = wz + ((α+1)/(1−s_z^(λ)))(wz − w*) that skips the second Hessian-inverse-vector product (Eqs. 8–15).
- The method achieves an ~82× wall-clock speedup over standard influence-function unlearning for subsequent correlated points (2.07 ms vs 169.54 ms, California Housing, Table III).
- The parameter-update error versus standard sequential unlearning is bounded by a polynomial-in-d quantity (Theorem V.1), empirically conservative by ~31–39× (Table IX).
- On CIFAR-10/ResNet-18 the method attains ToW 0.950 versus RUM(B)'s 0.920, and the authors present MIA 0.660 as demonstrating superior forgetting effectiveness.
- Hessian damping keeps the condition number near 5×10^4 (λ=0.01) versus 1.88×10^13 undamped (Table V), securing the Sherman–Morrison update numerically.
Strengths
Elegant, deployable closed-form update
The rank-1 Sherman–Morrison route (Eqs. 4–15) reduces subsequent correlated unlearning to O(d) scalar/vector operations, with a concrete algorithm (Algorithm 1) needing only cached w_z, w*, and the self-influence score s_z.
Above-average numerical-stability treatment
Condition-number analysis (Table V: κ drops from 1.88×10^13 undamped to ≈49,735 at λ=0.01), a Rayleigh–Ritz guard ensuring s_z ≠ 1, and a λ sensitivity sweep (Table X) with matched condition numbers.
Actionable safety analysis
Broad evaluation surface for the setting
Weaknesses
MIA result contradicts the headline claim
Theory–experiment gap
Sparse baseline coverage
Scalability overstated
No code release; internal inconsistencies
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This method paper targets approximate machine unlearning for *sequential removal of correlated points*, motivated by security operations (poisoned data in intrusion detection, spam filters) and GDPR erasure requests. The central claim: once point z has been unlearned with one expensive influence-function step (Eq. 2, damped Hessian H_λ = H_w* + λI), later points x similar to z need no new Hessian-inverse-vector product — a closed-form update w'x = wz + ((α+1)/(1−s_z^(λ)))(wz − w*) (Eq. 15) suffices, where α is the Pearson correlation of raw feature vectors and s_z^(λ) is a damped self-influence score. Ancillary claims: an error bound polynomial in dimension d (Theorems IV.1, V.1), numerical stability via damping, an 82× wall-clock speedup (Table III), and ~10⁻²-scale accuracy gains over baselines.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is correlation-guided fast machine unlearning?
It is an approximate unlearning scheme in which the first data point is removed with a standard influence-function update (one damped Hessian-inverse-vector product), and subsequent data points that are highly correlated — by Pearson correlation of their raw feature vectors — with that first point are removed via a cheap closed-form scaling of the already-computed update, avoiding repeated expensive Hessian computations.
How much faster is it than standard influence-function unlearning?
The paper reports roughly an 82× wall-clock speedup per subsequent correlated point on the California Housing regression task (2.07 ms versus 169.54 ms per point), reducing per-point complexity from O(d²) to O(d). The measurement is on a small 8-feature linear model.
Does the paper show better privacy or forgetting than prior methods?
Mixed. On CIFAR-10/ResNet-18 its tug-of-war score (0.950) edges out RUM(B) (0.920), but its membership-inference success rate (0.660) is higher — which the paper itself defines as worse — than RUM(A) (0.489) and RUM(B) (0.590). The abstract nonetheless presents the 0.660 figure as evidence of superior forgetting.
Is code available for this paper?
The paper does not mention any public code or data release. The experiments use standard public datasets (scikit-learn California Housing and Diabetes, a synthetic Gaussian mixture, MNIST, Fashion-MNIST, CIFAR-10, CIFAR-100, LFW), so a careful reimplementation is possible but requires re-deriving unreported protocol details.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Table VI is a dense multi-column baseline table; PDF extraction can misalign columns, so the exact attribution of individual values to MITR vs Hessian-free columns (and the caption–content tension over MITR on CNN rows) may partly reflect extraction rather than the paper. All other cited numbers were verified against the extracted text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.