Subgroup Membership Inference Audits of Differentially Private Synthetic Text
Yidan Sun, Viktor Schlegel, Srinivasan Nandakumar, Siew Kei Lam, Anil Anthony Bharath
cs.CR · cs.AI
Quick verdict
Formalizes subgroup-targeted MIA audits of DP synthetic text; rigorous, unusually honest about limits, but headline AUCs rest on oracle proxy selection.
Summary
The paper formalizes a pool-parameterized membership inference (MI) game for synthetic text releases, in which the target pool is an explicit audit parameter, and uses it to test whether differentially private (DP) generators protect high-risk subgroups. The audit covers 32 proxy attacks under three attacker-knowledge scenarios, four datasets (clinical, financial, legal), three generator families (DP-SGD fine-tuning, API-based prompting, activation steering; all Llama-3.2-1B), and five privacy budgets (∞, 4, 2, 1, 0.5). Non-private fine-tuning leaks subgroup membership strongly (best AUC 0.79–0.83; TPR 34–61% at 5% FPR); under every DP budget leakage persists above chance (AUC 0.55–0.65). Re-instantiated prior attacks (CANARY, DOMIAS) lose in 69/78 cells. Residual leakage concentrates: the top decile of records carries roughly 40% (up to 72%) of positive per-record vulnerability, and under DP high-risk pools stay more distinguishable than random controls. However, which records leak is mechanism-dependent, and no intrinsic attribute predicts vulnerability (all Holm-corrected nulls), implying per-release auditing. Key limitations: headline AUCs are hindsight best-proxy envelopes — the deployable fusion loses ~0.077 AUC; confidence intervals capture only negative-sampling variance; pools are small (16–71) and corpora modest (604–5,102).
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Subgroup membership remains inferable above chance at every tested DP budget (best-proxy AUC 0.55–0.65; TPR up to ≈18% at 5% FPR), while non-private fine-tuning leaks strongly (AUC 0.79–0.83, TPR 34–61%).
- Re-instantiated prior release-based attacks (CANARY, DOMIAS) systematically under-measure subgroup leakage, winning only 7–9 of 78 cells against the heterogeneous proxy family.
- Under DP, residual leakage is concentrated: the top decile of records carries roughly 40% (up to 72%) of positive per-record vulnerability (Table 19).
- DP's realized protection is uneven: high-risk pools stay more distinguishable than matched random controls under every DP budget, and a merged-pool audit confirms the record-level exposure difference.
- Which records leak is a property of the release mechanism rather than the record: vulnerability does not transfer across generators and no intrinsic attribute survives Holm correction, so risk must be audited per release.
Strengths
Clean, reusable formalization
The pool-parameterized game (Definition 1, §3.2) makes the target pool an explicit parameter and unifies high-risk, random-control, and composite-pool audits in one protocol; Remark 1 fixes the well-posedness problem of comparing within-pool games that normalize against different complements.
Well-controlled experimental design
32 proxies × 3 attacker scenarios × 4 datasets × 3 generator mechanisms on a shared Llama-3.2-1B backbone × 5 budgets; negative-candidate plans are fixed across generators and ε (§4.6), so cross-method differences reflect the mechanism rather than sampling luck.
Unusually candid statistical hygiene
Record-level adjudication experiment
Actionable and mechanism-relative conclusions
Weaknesses
Headline numbers are oracle-selected
No pool-draw variance
Record-level claim rests on two cells
Generator-setting deviation in a baseline mechanism
Attribute null is partly power-limited
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This preprint (arXiv 2609.09848; Imperial College London / NTU Singapore) addresses a real gap in synthetic-text privacy auditing: existing release-based membership inference (MIA) audits report average-case risk over randomly drawn records, which can hide risk concentrated in atypical subgroups. The authors formalize a pool-parameterized membership game (Definition 1, §3.2) in which the target pool is an explicit parameter — OUTLIER, RARE-label, RANDOM control, or a composite union (Remark 1) — so targeted versus untargeted leakage becomes directly comparable within one protocol. Five research questions (§4) structure the audit: whether subgroup membership leaks under DP (RQ1), how risk scales with attacker knowledge (RQ2), whether prior attacks suffice for subgroup auditing (RQ3), whether DP protects pool members uniformly (RQ4), and whether at-risk records are predictable (RQ5).
The central claims: (1) subgroup membership remains inferable above chance at every DP budget tested (best-proxy AUC 0.55–0.65, TPR up to ≈18% at 5% FPR, §5.1); (2) re-instantiated prior attacks (CANARY, DOMIAS) underestimate leakage — the proxy family wins 69/78 cells (§5.3, App. 7.7); (3) residual leakage concentrates — the top decile of records carries ~40% (up to 72%) of positive per-record vulnerability (§5.4, Table 19); (4) DP's protection is uneven, with high-risk pools staying more distinguishable than random controls, confirmed at record level via a merged-pool audit (§5.6, Tables 5–6); (5) which records leak is mechanism-bound, not record-bound (§5.5, App. 7.11).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Does differential privacy stop membership inference on synthetic text?
Not entirely, according to this audit. Even at every tested DP budget (ε = 4, 2, 1, 0.5), membership of high-risk subgroup records remains inferable above chance, with best-proxy attack AUC 0.55–0.65 and true-positive rates up to about 18% at a 5% false-positive rate. DP does cut average leakage substantially compared to non-private fine-tuning (AUC 0.79–0.83), but the residual leakage concentrates in a small subset of records — the top 10% of records carry roughly 40% of it.
What is a subgroup-targeted membership inference attack?
It is a privacy audit in which the records the attacker must decide about (the target pool) are chosen by an explicit rule — for example the most atypical or rarest-label records — rather than drawn at random. The paper's game treats the pool as a parameter of the audit itself, so leakage on high-risk subgroups can be compared directly against matched random-control pools under identical release conditions.
Which synthetic text generators were audited in this study?
Three mechanism families, all built on Llama-3.2-1B: DP-Transformers (DP-SGD fine-tuning, enforcing privacy during training), Aug-PE (API-based prompting with privatized corpus statistics), and EPSVec (activation steering via a privatized dataset vector). Each was tested at ε ∈ {∞, 4, 2, 1, 0.5} on four corpora spanning clinical (N2C2'08, PSYTAR), financial (DMSAFN), and legal (EURLEX) text.
Can you predict in advance which records will leak membership?
In this study, no. Five release-independent attributes (length, embedding outlierness, near-duplication, rare-n-gram density, entity/numeral density) showed no Holm-corrected correlation with per-record vulnerability across 210 attribute–pool tests, and vulnerability rankings did not transfer across generators (correlations near zero, Table 23). The authors conclude that record-level risk is a property of the release mechanism, so each synthetic release must be audited individually.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure interiors (Figs. 2–5) contain garbled axis and annotation text from PDF extraction; numeric conclusions were taken from captions, tables, and body text, which are intact.
- Mathematical notation in some table captions (e.g., the Table 5 interval formula) is garbled by extraction, but the surrounding prose is readable.
- Exact per-proxy feature names are deferred to a companion CSV not included in the extracted text; proxies are identified only at family level (Table 8).
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.