QuanText: Protecting Dataset-Level Secrets in Textual Data Sharing
Shuaiqi Wang, Zinan Lin, Giulia Fanti
cs.LG · cs.CR
Quick verdict
QuanText hides dataset-level proportions in text via randomized attribute distributions + LLM rewriting; solid SML theory, but empirics rely on one weak attack.
Summary
The paper targets dataset-level privacy in text release: a published corpus can reveal global properties (e.g., proportion of diabetic patients), and differential privacy protects records but largely cancels out on such aggregates. The authors propose QuanText, a training-free, LLM-agnostic release mechanism. It factorizes the joint distribution of utility-relevant attributes with a Chow-Liu tree learned from public auxiliary data, constructs gamma candidate distributions per factor (Dirichlet sampling plus greedy farthest-point selection), then uniformly picks one of the k candidates closest in total variation to the private empirical distribution. Samples are Hungarian-matched to target attribute combinations and rewritten by an LLM using attribute-related snippets from the original text. Building on Statistic Maximal Leakage (SML), they prove a bound of log(gamma/k)^(1+sum psi) plus an additive log l term for content-style correlation, yielding a worst-case attack success bound. On Tweet Stance and ChatDoctor, against Private Evolution, DP fine-tuning (epsilon 1-4), raw release, and subsampling, QuanText with gamma/k <= 8 attains higher attack MAE and higher aggregate utility; lowering gamma/k from 12 to 2 raises MAE from ~0.01 to ~0.1 at modest utility cost (0.85 to 0.77). Key limitations: empirical privacy is measured only via an LLM labeling attack despite an unbounded-attacker theory; the guarantee rests on an unverifiable LLM-faithfulness assumption and an unestimable correlation parameter l; no downstream-task utility evaluation; headline results lack error bars or significance tests.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Differential privacy, designed for individual records, provides weak protection for dataset-level aggregate properties in text release (Sec. I).
- QuanText, which randomly selects among candidate attribute distributions and LLM-rewrites samples to match, provably satisfies an SML bound of log(gamma/k)^(1+sum psi_pi(i)) + log l under idealized conditions (Thm. IV.3).
- The SML bound yields a worst-case attacker success probability of at most alpha*(gamma/k)^(1+sum psi_pi(i)) for any prior-knowledge success rate alpha (Prop. IV.4).
- Empirically on Tweet Stance and ChatDoctor, QuanText with candidate dilution gamma/k <= 8 achieves both higher attack MAE and higher aggregate utility than Private Evolution and DP fine-tuning across epsilon in {1,2,3,4} (Sec. V-D).
- Results are robust to the specific (gamma, k) values at fixed ratio (Fig. 5) and to the choice of attack backend model (Table I).
Strengths
Targets an under-served privacy notion with a practical defense
While prior defenses target classifiers or record-level DP (Sec. II), QuanText is a model-agnostic, training-free defense at the data-release stage that explicitly handles leakage through correlated attributes, which defeats secret-only perturbation like [10]; the pipeline (Chow-Liu factorization, candidate construction, Hungarian matching, snippet reuse) is concretely specified in Alg. 1-2.
End-to-end theoretical guarantee tied to mechanism knobs
Thm. IV.3 bounds SML by log(gamma/k)^(1+sum psi_pi(i)) + log l with an operational corollary (Prop. IV.4: attack success <= alpha*(gamma/k)^(...)); App. B derives it from SML post-processing, and the parameters gamma, k, and tree arity map directly to tunable design choices.
Favorable empirical privacy-utility trade-off
Ablations beyond the headline comparison
Code availability
Weaknesses
Empirical privacy rests on a single weak attack
Guarantee hinges on unverified assumptions and an unestimable constant
Asymmetric baseline comparison
No downstream-task utility evaluation
Thin statistical reporting and small samples
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is dataset-level privacy and how is it different from differential privacy?
Dataset-level (distribution-level) privacy protects global properties of a whole dataset, such as the proportion of records tied to a diagnosis, gender, or political stance. Differential privacy instead protects individual records; because its noise is typically zero-mean, it largely cancels out in aggregates, leaving such proportions close to their true values and thus offering weak protection for these global secrets.
What is Statistic Maximal Leakage (SML)?
SML is a privacy measure that bounds how much observing released data can improve an attacker's probability of correctly guessing a specified secret function of the dataset, in the worst case over data priors and arbitrary attack strategies. If SML is at most eta, any attack succeeds with probability at most alpha times e^eta, where alpha is the success rate from prior knowledge alone. SML is robust to post-processing and supports adaptive composition.
Does QuanText require training a model or a specific LLM?
No. QuanText is training-free and LLM-agnostic: it uses an off-the-shelf LLM (Llama3.1-8B-Instruct in the experiments) only to label attributes of each text, extract attribute-related snippets, and rewrite each sample so the released dataset matches a randomly selected target attribute distribution.
How well does QuanText perform compared to DP synthetic data baselines?
On the Tweet Stance and ChatDoctor datasets, when the candidate-dilution ratio gamma/k was at most 8, QuanText achieved both higher attack error (MAE) and higher aggregate utility than Private Evolution and DP fine-tuning across privacy budgets epsilon = 1 to 4. Lowering gamma/k from 12 to 2 raised attack MAE from about 0.01 to 0.1 while aggregate utility fell only from about 0.85 to 0.77.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 3-5 appear in the extracted text only as axis labels and ranges; exact curves and data points could not be verified, so quantitative claims are taken from the surrounding prose.
- Prompt templates, the utility-metric rescaling procedure, and any appendix material beyond Alg. 2 and the Thm. IV.3 proof are not present in the extracted text.
- This is an arXiv v1 preprint (Sep 2026); peer-review status is unknown.
- Minor extraction artifacts: the text repeatedly cites 'Thm. IV.2' where Assumption IV.2 appears to be meant, and exponent placement in Thm. IV.3 may be garbled by extraction.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.