SoK: Privacy Attacks on Machine Learning via Explainable AI
Abdullah Caglar Oksuz, Anisa Halimi, Erman Ayday
cs.CR · cs.LG
Quick verdict
SoK systematizing 25 XAI-enabled privacy attacks via a 5-path acquisition taxonomy; clear, honest, and useful, though corpus is small and evidence qualitative.
Summary
This SoK systematizes 25 studies that weaponize machine-learning explanations for model extraction, membership inference, and model inversion (treating attribute inference as partial inversion). The authors' core move is separating model knowledge from explanation acquisition: they argue the conventional black-box/white-box split obscures what explanation signal actually reaches an adversary, and introduce five acquisition paths — target-released (P1), attacker-derived (P2), secondary disclosure of another user's report (P3), privileged access (P4), and released global artifacts (P5) — coding each study across threat model, explanation signal, auxiliary knowledge, query budget, modality, and defenses. Key syntheses: gradient oracles collapse extraction cost (95% MNIST accuracy with 10 gradient queries vs 1,000 label queries; MEGEX matches DFME's 81.13% CIFAR-10 accuracy at 2.5M vs 20M queries); counterfactual-distance membership inference is heterogeneous (HELOC AUC 0.679, TPR 5.13% at 1% FPR; several Adult settings near random); Shapley reports enable near-exact query-record inversion for linear models; and defenses are path-specific — DP-SGD does not protect query records, and MemGuard leaves explanation-channel attack AUC at 0.872–0.885. Limitations: the corpus is small and non-PRISMA, coverage skews to local explanations in tabular/image settings with LLM/text explanations barely covered, no new experiments are run, and the marginal-leakage quantity Δ_XAI remains conceptual because cross-study metrics are incomparable.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Model knowledge and explanation disclosure are orthogonal dimensions, so black-box/white-box labels alone underspecify an explanation-assisted attack surface (§4.1, Eq. 3).
- Five acquisition paths (P1 target-released, P2 attacker-derived, P3 secondary disclosure, P4 privileged access, P5 released global artifacts) determine whether attacks survive defenses; disabling an explanation endpoint does not stop P2 or P3 attacks.
- No explanation family is uniformly unsafe and no defense is uniformly effective; risk is determined by the exposed signal, acquisition path, protected asset, and attacker's prior knowledge (§8, §12).
- DP training protects training records only, not query-record inversion or model confidentiality; each defense must be matched to the acquisition path and asset it actually covers (§9.1, §9.5).
- Explanations measurably reduce attack cost and expose new signals: 10 gradient queries reach 95% MNIST extraction accuracy vs 1,000 label queries (§5.1), and exact Shapley values permit error-free linear-model reconstruction (§7.2).
Strengths
Genuinely new acquisition taxonomy
The P1–P5 paths (§4.2) separate how an explanation is obtained from how much of the model is known (Eq. 3), directly fixing the black-box/white-box comparability problem the paper diagnoses; the contrast with Allana et al.'s 57-study scoping review and Nguyen et al.'s method taxonomy is explicit (§2.2, §11).
Unusually honest reading of primary results
The SoK repeatedly flags where gains are not attributable to XAI alone: Liu et al.'s explanation-free ablation also improves MIA (§6.3, §8.3), MEGEX's absolute budgets of 2.5–20M queries are called out as not cheap exploitation (§5.1), and Pawelczyk's near-random Adult/Diabetes configurations are reported alongside the leaking HELOC setting (§6.2).
Actionable defense mapping
Concrete community deliverables
Weaknesses
Small, non-systematic corpus
No quantitative synthesis
Coverage gap in the most deployment-relevant modality
Evidence inherited at face value
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This SoK covers 25 studies in which an explanation artifact (gradients/saliency, IG/Grad-CAM/LRP, LIME, SHAP, counterfactuals/recourse, influence, global surrogates, GNN explainers — Table 1) is the offensive primitive for model extraction (13 studies), membership inference (9), and model inversion/attribute inference (4; one study spans two families). Its positioning against prior surveys — Allana et al.'s 57-study TMLR scoping review and Nguyen et al.'s explanation-type taxonomy (§2.2, §11) — is credible: the unit of analysis is the acquisition path, not the explanation method. Central claims (§4, §8, §12): (i) model knowledge and explanation disclosure are orthogonal, so black-box/white-box labels are underspecified; (ii) five acquisition paths P1–P5 (§4.2) determine whether attacks survive endpoint hardening; (iii) no explanation family is uniformly unsafe and no defense uniformly effective. Claim (iii) is a universal negative over an incompletely sampled 25-study space and should be read as the organizing thesis, not an established result.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What are the five explanation-acquisition paths in this SoK?
P1 target-released (the service returns the explanation for your query), P2 attacker-derived (the attacker reconstructs explanations from predictions alone), P3 secondary disclosure (another user's explanation report is leaked or shared), P4 privileged access (training or debugging information), and P5 released global artifacts (a public surrogate model). The key point is that these paths are independent of how much the attacker knows about the model itself, so defenses must be matched to the path, not just to the model.
Do explainable AI features make machine learning models easier to attack?
Yes, but conditionally. This paper finds gradient explanations can cut model-extraction cost by orders of magnitude (95% accuracy on MNIST with 10 gradient queries versus 1,000 label-only queries), and Shapley explanation reports can enable near-exact reconstruction of a user's input for linear models. However, counterfactual-based membership inference is strong only in some settings (HELOC: AUC 0.679; several Adult configurations near random). The overall conclusion is that no explanation type is uniformly unsafe — risk depends on which signal is exposed, how it is acquired, and what the attacker already knows.
Which defenses protect against explanation-based privacy attacks?
No single defense covers everything. Differential privacy (DP-SGD) protects training-data membership but not the records users submit at inference time and not model confidentiality. Adding noise to explanations helps only if it removes the attack-relevant statistic — under the confidence-perturbation defense MemGuard, explanation-channel membership attacks still achieved AUC 0.872–0.885. Query rate limiting addresses only attacker-queried explanations, while leaked explanation reports require conventional access control, encryption, and retention limits.
Is this paper a survey or does it introduce new attacks?
It is a systematization of knowledge (SoK): it organizes 25 previously published attack and defense studies under a new five-path acquisition taxonomy and a ten-item reporting checklist, but introduces no new attack, dataset, trained model, or experiments, and provides no code artifact (Appendix A).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- PDF extraction reflows the cell text of Tables 1–5 across line breaks; all quoted numbers appear verbatim in the text, but exact cell boundaries were inferred from layout.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.