Canaries in the Bank: Auditing User-Level Privacy in Private Evolution
Sai Aparna Aketi, Enayat Ullah, Shripad Gade
cs.CR · cs.AI · cs.LG
Quick verdict
First protocol-aware audit of Private Evolution: careful stats show natural-text attacks hit ~40% of DP bound, nonces ~67%; minor cross-table inconsistencies.
Summary
The paper introduces the first empirical privacy audit of Private Evolution (PE), a federated DP synthetic-data mechanism in which users vote over a server-proposed candidate bank. Adopting an honest-but-curious server threat model, the authors replace about 1% (64 of 8,192) of the shared candidate bank with probes crafted from a known canary user, then apply a likelihood-ratio hypothesis test that marginalizes over hidden participation to output a confidence-bounded lower bound on the empirical epsilon. Eight pre-declared attack variants span passive observation, exact-copy injection, paraphrases (with norm- and background-aware greedy selection), and high-entropy nonce probes. On Yelp and Sentiment140 with MiniLM embeddings (q=0.1, σ=1, δ=1e-5, εtheory=1.695), natural-text attacks recover at most ~40% of the theoretical budget (paraphrase-μeff: 0.560/0.682), while nonce attacks reach ~1.11 (~67%), approaching the derived separation ceiling μeff ≤ 1/σ. A six-round multi-round audit on Yelp shows utility (MAUVE) is preserved while composed empirical epsilon tracks the PRV accountant. Results are statistically careful: family-wise error control across five canaries per variant, frozen calibration splits, and Clopper–Pearson bounds with 1M–5M trials. Limitations: text-only modality, no code release, unexplained inconsistencies between Table 2 and the Appendix C Bank A numbers, and exclusive reliance on lower bounds.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A protocol-following honest-but-curious server can inject roughly 1% candidate-bank probes that create a detectable membership signal while otherwise leaving PE unchanged.
- Natural-text attacks realize only 15–40% of the theoretical user-level DP budget (εlower ≤ 0.682 vs εtheory = 1.695 at q=0.1, σ=1, δ=1e-5), while nonce probes reach ~1.11 (~67%).
- The achievable separation is capped by μeff² ≤ 1/σ², explaining why nonce attacks approach but do not reach the PRV accountant bound.
- The audit composes correctly across adaptive PE rounds (chain rule over conditional likelihood ratios) and preserves synthetic-data utility (MAUVE) over six rounds on Yelp.
- Empirical ε is insensitive to encoder capacity across MiniLM-L6, MiniLM-L12, and MPNet-base-v2 (Table 3).
Strengths
Novel audit target with clear positioning
First protocol-aware audit of PE's shared candidate bank (Related Work section): the adversarial object is one public bank sent unchanged to all clients, distinct from CANIFE's client updates or SnapAudit's ICL snapshots.
Disciplined inference machinery
Banks frozen before the hidden inclusion draw, calibration split separated from final evaluation, exact mixture likelihood ratio marginalizing hidden participation (Eq. 4, Appendix E.5), and family-wise α=0.05 across five canaries with Clopper–Pearson bounds at γ=0.0025.
Theory-experiment linkage
Correct multi-round composition
Weaknesses
Internal inconsistencies between headline tables
One-sided evidence
Unvalidated Gaussian approximation
No code or artifacts
Multiplicity scope
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper introduces a protocol-aware privacy audit for Private Evolution (PE), the family of federated DP synthetic-data mechanisms in which clients vote over a server-proposed candidate bank and a noise-calibrated DP histogram decides survival (Section 1, Algorithm A.1). The threat model is an honest-but-curious server that commits, before the hidden inclusion draw, to a bank in which roughly 1% of entries (r=64 of p=8,192) are probes derived from a known canary. Central claims: (i) protocol-valid bank manipulation creates a measurable membership signal; (ii) natural-text attacks recover only 15-40% of the theoretical budget (Table 2: paraphrase-μeff 0.560 on Yelp, 0.682 on Sentiment140 vs εtheory=1.695 at q=0.1, σ=1, δ=1e-5); (iii) high-entropy nonce probes approach the mechanism's separation ceiling (nonce-norm 1.110/1.088, ~67%), consistent with the derived bound μeff² ≤ 1/σ² (Eq. 20); (iv) the audit composes across six adaptive PE rounds on Yelp (Figure 2, εtheory 1.345→2.000) while preserving MAUVE utility.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Private Evolution (PE) privacy auditing?
PE is a federated technique that generates synthetic data: clients vote for candidate samples near their private records, and a differentially private histogram decides which candidates survive. Auditing here means empirically checking whether a curious server can detect a specific user's participation while faithfully following the protocol, by planting probe entries in the shared candidate bank.
How much of the theoretical DP budget do attacks on Private Evolution recover?
In this paper, at sampling rate q=0.1, noise multiplier σ=1, and δ=1e-5 (theoretical ε=1.695), attacks using natural text recovered at most about 40% of the budget (empirical ε up to 0.682 on Sentiment140), while attacks with high-entropy nonce probes reached about 1.11, roughly 67% of the theoretical bound.
Is the nonce attack a realistic privacy threat to Private Evolution?
No. The authors use nonce probes to calibrate the mechanism's ceiling on detectable signal, because nonce strings attract almost no background votes from real users. The realistic case — manipulating the candidate bank with natural text — recovered far less of the privacy budget.
Is code available for this paper?
The provided text contains no code, dataset, or artifact release statement, so the audit would need to be reimplemented from the algorithms and hyperparameters given in the appendices (Algorithms A.1 and B.1, Appendix E).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Inline math is OCR-garbled in places (e.g., the clipping term in Eq. 1 and the scores in Eq. 6), but all formulas remain interpretable from context; the text runs complete through Appendix E.8 with no truncation observed.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.