PAC-Private Autoregressive Generation: Calibrating Noise to Ensemble Disagreement
Mina Mirzadehsarcheshmeh, Amir Keyvan Khandani
cs.LG · cs.AI
Quick verdict
First PAC-private autoregressive LLM generation with careful theory and honest evaluation; evidence limited to one corpus and GPT-2-small.
Summary
The paper extends PAC privacy — instance-based noise calibration that bounds an adversary's inference advantage via mutual information — from classification to autoregressive language-model generation. From a 2,048-article WikiText-103 universe the authors build 128 overlapping 'worlds' (each record in exactly 64), train one LoRA adapter per world on frozen GPT-2-small, and treat the realized world as the secret. Per token, the public model fixes a Top-200 candidate set, all worlds vote, and posterior-weighted disagreement sets the Gaussian noise; unanimous positions cost nothing. They prove I(S;Y1:T) <= I(S;H_T) <= bT, handling token-only release, singular vote covariance, self-generated queries, and shared-randomness Gumbel decoding. At b=2^-32 the mechanism retains 74.3% (greedy) / 84.4% (Gumbel) of the fine-tuning gain, with membership-inference success provably <=51.08% after 10^6 tokens; posterior-entropy estimates put measured leakage near 17% of the charged budget. Against a validated PMixED reimplementation at matched provable bounds, PAC retains 98% of non-private headroom across 10^2-10^6 tokens versus at most ~56% for PMixED. Canary experiments separate inference privacy from content protection. Main limitations: one corpus, one base model, one world partition, and empirical attacks that read the curator's own posterior rather than an independent adversary.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- This is the first instantiation of PAC-private prediction for autoregressive generation, with an end-to-end guarantee I(S;Y1:T) <= I(S;H_T) <= bT covering token-only release, self-generated queries, singular vote covariance, and coupled Gumbel decoding.
- At per-token budget b=2^-32 the mechanism retains 74.3% (greedy) / 84.4% (coupled Gumbel) of the fine-tuning gain on WikiText-103/GPT-2-small, with membership-inference success provably bounded at 51.08% after 10^6 released tokens.
- Measured leakage, estimated directly from posterior entropy, is roughly 17% of the charged fixed per-token budget and stays near-constant across budgets and horizons.
- At matched provable membership-inference bounds on the same data universe, PAC retains 98% of non-private headroom across horizons 10^2 to 10^6 tokens while PMixED retains at most ~56%, with no crossover under either accounting convention.
- Inference privacy does not imply content protection: at b=2^-16 canary membership advantage is statistically indistinguishable from zero while the memorized canary is still emitted at essentially the same rate.
Strengths
Complete theory for the new setting
Theorem 4 plus Lemmas 2-4 and Props 2-3 each address one genuinely new autoregressive obstacle (token-only release, singular covariance with pseudoinverse belief updates, self-generated queries, coupled Gumbel decoding under explicit coin conditions C1-C3); Remark 1 and Appendix A.5 correctly show the bound cannot be routed through R1:T when the voted set depends on public coins.
Attack-agnostic direct leakage measurement
Proposition 4 turns the enumerable secret space into a verification tool: 15 session cells give measured I(S;H_T) at 17.0-18.8% of charged budget, with three independent T=4096 estimates at 17.311/17.209/17.318% (SD 0.061pp) - stronger than attack-based probing, which only lower-bounds leakage.
Validated, convention-aware PMixED comparison
Two field-level diagnostic findings
Honest reporting of limits of its own results
Weaknesses
Single-partition evidence
Empirical attacks are upper envelopes, not adversaries
Small-scale, thin-margin utility regime
Noise-neutrality may not generalize
No code release; inherent serving cost
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper claims the first extension of PAC-private prediction from classification (Zhu et al.) to autoregressive language-model generation. The construction: a 2,048-article WikiText-103 universe (plus 16 canaries) is split into m=128 overlapping worlds, each record in exactly 64; one LoRA adapter (r=8) per world sits on a frozen GPT-2-small base, and the realized world is the secret. Per released token, the public model fixes a Top-200 candidate set, all worlds cast one-hot votes inside it, and Gaussian noise calibrated to the posterior-weighted vote covariance protects the realized world's vote; only the argmax token is released, with a public fallback once the budget is exhausted. Five claims: (1) an end-to-end guarantee I(S;Y1:T) <= I(S;H_T) <= bT surviving token-only release, self-generated queries, singular covariance, and coupled Gumbel decoding (Theorem 4; Lemmas 2-4; Props 2-3); (2) 74.3%/84.4% retained fine-tuning gain at b=2^-32 with provable MIA success <=51.08% after 10^6 tokens; (3) measured leakage of roughly 17% of the charged budget; (4) 98% headroom versus <=55.7% for PMixED at matched bounds, no crossover; (5) inference privacy does not imply content protection.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is PAC-private autoregressive generation?
A scheme for serving LLM text in which, at every generated token, Gaussian noise is calibrated to how much 128 privately-trained ensemble variants ('worlds') disagree, provably bounding what an output observer can infer about which private data split was used - via mutual information rather than worst-case differential privacy.
How much utility does the privacy mechanism cost?
Very little in this paper's setting: at per-token budget 2^-32, teacher-forced next-token accuracy drops only ~0.37 points (greedy), retaining 74.3% of the fine-tuning gain (84.4% with Gumbel decoding). Accuracy is flat across budgets 2^-4 to 2^-32 because ~84% of positions are unanimous and receive no noise.
Does low membership-inference success mean memorized text cannot leak?
No. The paper shows that at b=2^-16 a forcibly memorized canary has a membership advantage statistically indistinguishable from zero, yet the canary is still emitted at essentially the same rate regardless of membership. Inference privacy and content protection are separate guarantees needing separate mechanisms.
How does this method compare to PMixED?
On the same 2,048-article WikiText-103 universe at matched provable membership-inference bounds, the PAC mechanism retains ~98% of its non-private accuracy headroom from 10^2 to 10^6 tokens, while PMixED falls from ~56% to 1-3% depending on horizon and accounting convention, with no crossover at any tested horizon.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text is a sampled excerpt: marked sampling gaps omit parts of Sec 3.3, parts of Secs 5.2-5.3, Table 9's numeric rows, and references 28-34; all numbers cited in this review appear in the visible text.
- No code-availability statement appears anywhere in the provided excerpt; codeAvailable=false reflects that absence in the excerpt, not a verified absence in the full paper.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.