DenMark: Robust Semantic Watermarking for Diffusion Language Models
Tianhao Ma, Weihao Xuan, Dong-Dong Wu, Farshid Nooshi, Takashi Ishida, Gang Niu, Naoto Yokoya, Masashi Sugiyama
cs.CL
Quick verdict
Claims first semantic watermark for diffusion LMs, best in all 48 test cells; strong, but 12x generation cost and weak expansion TPR temper the win.
Summary
The paper introduces DenMark, presented as the first semantic watermarking framework designed for diffusion language models (DLMs). Because DLM denoising updates tokens in arbitrary order, the complete sentence-level candidate scoring used by autoregressive semantic watermarks is impractical; DenMark instead partitions output into fixed token regions and, at each denoising step, samples candidate local updates, completes each with one-step rollouts, and commits the update maximizing a key-dependent semantic score over random directions in a contrastively fine-tuned E5 embedding space. Detection scans candidate unit sizes (12-37), calibrates each against 30,000 C4 passages, and applies Bonferroni correction, yielding proven false-positive control (Proposition 4.1). Across four DLMs, three WaterBench datasets, and four GPT-4o-mini semantic attacks, the paper reports the best value on all four detection metrics in all 48 backbone-dataset-attack combinations, averaging 63.55/71.89/87.45% TPR at 0.5/1/5% FPR and 0.9693 AUC versus 35.16/41.90/61.80% and 0.8915 for the strongest baseline. Main limitations: detection under expansion attacks is weak (TPR@0.5% of 27.76-36.33% on Finance across backbones), generation costs 0.654 s/token (about 12x clean decoding and 2.8-5.5x baselines), downstream GSM8K accuracy drops about 10.5 points on LLaDA-8B, and no code release is stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- DenMark is the first semantic watermarking framework designed for diffusion language models, injecting key-dependent evidence through rollout-scored local updates during denoising.
- Calibrated scanning over unit sizes 12-37 with Bonferroni correction provides provable false-positive control under attack-induced boundary shifts (Proposition 4.1).
- DenMark achieves the best value on all four detection metrics in all 48 backbone-dataset-attack combinations, averaging 63.55/71.89/87.45% TPR at 0.5/1/5% FPR (AUC 0.9693) versus 35.16/41.90/61.80% (0.8915) for the strongest baseline.
- The final watermark score gain decomposes exactly into accumulated per-step advantages (Proposition 5.1), and the empirical cumulative advantage is positive at decoding end for both tested backbones.
- Generation quality stays comparable to baselines (average delta mean-log-PPL +0.34 and judge -0.41, both the best among evaluated methods, Table 10).
Strengths
Clear problem identification and first-mover design
Figure 1 precisely articulates why sentence-level candidate-reject-commit semantic watermarks (SemStamp, k-SemStamp, PMark) fail under any-order DLM denoising; the rollout-as-lookahead adaptation (Sec. 3.2) is a natural and well-argued answer, validated by the exact telescoping decomposition in Proposition 5.1 and its empirical check (Fig. 2: final cumulative advantage 0.0387 on LLaDA-8B, 95% CI [0.0251, 0.0527]).
Statistically careful detection with a correctness proof
Sec. 4 uses empirical right-tail p-values with the +1 correction (Phipson & Smyth), per-size calibration on 30,000 C4 passages, and Bonferroni over the scan set G={12..37}; Proposition 4.1 (proof in App. G.1) establishes FPR control without any independence assumption across scanned sizes — above the watermarking literature norm.
Unusually broad and honest evaluation
Complete implementation specification
Weaknesses
Expansion attacks expose weak absolute detection
No variance quantification behind the 48/48 sweep
Reasoning capability regression is under-discussed
No adversary model targeting the semantic detector itself
12x generation overhead versus 3-6x for baselines
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
DenMark (arXiv:2609.14257) targets semantic watermarking for masked diffusion language models (DLMs). Three contributions are claimed: (1) the first semantic watermarking framework designed for DLMs, motivated by the mismatch between sentence-level candidate-reject-commit watermarks (SemStamp, k-SemStamp, PMark) and any-order denoising (Figure 1); (2) a rollout-guided algorithm that scores local token updates by completing the active 25-token unit with temporary rollouts; (3) calibrated detection via scanning unit sizes with Bonferroni-corrected empirical p-values. The headline empirical claim is dominance: the best value on all four detection metrics (TPR@0.5/1/5% FPR, AUC) in all 48 backbone-dataset-attack combinations across four DLMs (LLaDA-8B/1.5-8B/2.0-mini, Dream-7B), three WaterBench datasets, and four GPT-4o-mini semantic attacks.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is DenMark and how does it watermark diffusion language models?
DenMark is a semantic watermarking method for diffusion language models (DLMs). During iterative denoising it splits the output into fixed 25-token regions; for each candidate token update it temporarily completes the region with one-step rollouts and keeps the update whose completed text best aligns with key-dependent random directions in a fine-tuned sentence-embedding space. Repeating this across steps accumulates a statistical signal that a detector holding the same key and encoder can recover.
How robust is DenMark to paraphrasing attacks?
According to the paper, very robust for most edits: averaged over 48 model-dataset-attack combinations it detects 71.89% of watermarked texts at 1% false positive rate (63.55% at 0.5%), versus about 41.90% for the best token-level baseline. The weak spot is expansion-style rewriting, where detection at 0.5% FPR drops to roughly 28-36% on the Finance dataset across the four tested models.
How much does DenMark slow down text generation?
The paper reports 0.654 seconds per output token, versus 0.054 for unwatermarked decoding (about 12x slower) and 0.070-0.236 for the evaluated watermark baselines. The cost comes from scoring 16 candidates with multiple rollouts at every denoising step; the authors position this as an acceptable trade-off for robustness-oriented use.
Does the watermark degrade output quality?
Modestly on average: mean log-perplexity rises by +0.34 and GPT-4o-mini judge scores fall by 0.41 points relative to clean outputs, both the smallest deviations among the evaluated watermark methods. However, on downstream benchmarks the GSM8K math score of LLaDA-8B drops about 10.5 points (59.44 to 48.98), the largest math degradation among the compared watermarks.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 2-8 are present only as extracted text and captions; visual trends (e.g., DIPPER degradation curves, sensitivity plots) could not be independently verified.
- A portion of Section 5 (the hybrid-trajectory setup) and parts of Appendices E/F were truncated in extraction, including one row of Table 7 (LLaDA-8B document back-translation).
- No code or data availability statement appears in the extracted text; one may exist in parts not captured.
- arXiv ID and several cited venues (ICLR/ICML/ACL 2026) are stated as given in the source text and were not independently verified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.