What Breaks Local Watermarks? A Robustness Benchmark for Local Invisible Image Watermarking
Kai Yao, Bence Szil\'agyi, Sebesty\'en Kamp, M\'at\'e Po\'or, M\'at\'e Szilveszter, Matyas K. Zsoldos, Marc Juarez
cs.CV · cs.AI · cs.CR
Quick verdict
All five local watermarking methods in this rigorous first benchmark fail under geometric or generative edits; statistics and baseline fairness leave gaps.
Summary
The paper presents the first systematic robustness benchmark for local invisible image watermarking, evaluating five methods (MaskWM, WAM, plus OmniGuard, TrustMark, and PixelSeal adapted from global watermarking) across 55 non-adaptive transformations in four categories (signal distortions, alignment changes, indirect and direct local edits), on four datasets (800 images per method), each with and without SyncSeal synchronization. It measures payload recovery (bit accuracy plus a binomial evidence score that normalizes payload length), localization IoU for the two pixel-map methods, and clean-setting fidelity. All methods fail under some transformation. MaskWM leads on recovery (clean bit accuracy 0.973, evidence score 17.08) and localization (IoU 0.902) but has the lowest fidelity (PSNR 49.48 dB). Only MaskWM stays above the p=10^-3 evidence threshold across all five signal-distortion subcategories, while geometric misalignment and generative inpainting/outpainting drive nearly all methods to random decoding. A control restoring pixels outside the edit mask shows Stable Diffusion inpainting breaks watermarks via whole-image reconstruction rather than edit-region overlap; recovery inversely tracks Canny edge density. SyncSeal selectively helps affine transformations but costs 7-19 dB PSNR and can hurt PixelSeal. Limitations include error bars computed over only four dataset means without inferential statistics, adaptation-induced uncertainty for three baselines, a single generative pipeline, and 512x512 normalization of all inputs.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- No evaluated local watermarking method is robust across all 55 transformations; MaskWM offers the strongest payload recovery (clean evidence 17.08) and localization (IoU 0.902) but the lowest clean image fidelity (PSNR 49.48 dB).
- Robustness depends strongly on transformation type: the strongest methods tolerate signal distortions, while geometric misalignment and generative local edits such as inpainting and outpainting can drive payload recovery to random decoding.
- Inpainting breaks local watermarks through whole-image reconstruction by the diffusion pipeline rather than overlap with the watermark region, as shown by the outside-restored control restoring MaskWM's evidence score to 14.62-17.45.
- Payload recovery and localization can fail independently (e.g., MaskWM retains IoU 0.901 under vertical flip while bit accuracy is 0.502) and must be evaluated as separate axes.
- Because all benchmark transformations are non-adaptive, each observed failure constitutes a practical scheme-agnostic black-box removal attack, implying current methods are not ready for reliable production provenance.
Strengths
Genuine gap filled with a unified taxonomy
Section 2.2 shows prior local-watermarking robustness evidence came only from method-specific evaluations (WAM's centered-rectangle composites; MaskWM's ten valuemetric plus three geometric distortions). The four-category taxonomy of 55 transformations (Section 3.4, Table 3) adds underrepresented indirect/direct local edits, including inpainting, outpainting, and splicing.
Principled cross-method metric
The evidence score -log10(p_random) from the binomial tail under random decoding (Section 3.5) makes the 32-256 bit payload lengths comparable; e.g., it reveals PixelSeal (adapted, 256-bit) beats native-local WAM (32-bit) in the clean setting (6.73 vs 3.39), which raw bit accuracy would obscure.
Mechanistic failure analysis, not just a leaderboard
Recovery and localization cleanly dissociated
Unusually transparent protocol
Weaknesses
Uncertainty reporting is thin for the breadth of claims
Three of five baselines are adapted, so negative results may be adaptation artifacts
Recovery rankings are not fidelity-matched
Generative-edit findings rest on a single pipeline and fixed resolution
Single region per image limits ecological validity
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Which local image watermarking method is the most robust?
According to this benchmark of five methods under 55 image transformations, MaskWM is the strongest overall: it is the only method whose payload recovery stays above the statistical evidence threshold across all five signal-distortion subcategories, and it reaches the best localization accuracy (IoU 0.902 in the clean setting). The caveat is that it also introduces the largest image-quality loss (PSNR 49.48 dB versus 62.55 dB for WAM).
Can AI inpainting remove local image watermarks even if it does not touch the watermarked region?
Yes. The paper shows that running a Stable Diffusion inpainting pipeline on a region unrelated to the watermark drives nearly all methods' payload recovery to random decoding. A control experiment that restores all pixels outside the edited area recovers performance almost fully, proving the damage comes from the pipeline subtly regenerating the whole image, not from overwriting the watermark itself.
Does adding a synchronization module like SyncSeal fix watermark robustness to rotations and crops?
Only partly. SyncSeal substantially improves payload recovery for affine transformations (for example +12.36 evidence score on shearing for MaskWM), but it fails on some misalignments such as vertical flip, it degrades image quality by 7 to 19 dB PSNR across the tested methods, and for PixelSeal it actually reduces recovery in the clean setting and several transformation categories.
Are current local watermarks ready for real-world provenance or copyright protection?
The authors argue no. Because every transformation tested is a simple, non-adaptive operation that requires no knowledge of the watermarking scheme, each failure case doubles as a practical black-box removal attack. Combined with weak performance on small regions and the fidelity-recovery trade-off, the benchmark concludes that no evaluated method is ready for reliable production deployment.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-8 could not be visually inspected; numeric claims rely on values stated in the text and figure captions.
- The metadata abstract is truncated mid-sentence, but the full-text section contains the complete abstract, so no substantive information was lost.
- Appendix Table 4 parameters were extracted as flattened text; minor formatting ambiguity in individual parameter values is possible.
- This is an arXiv preprint accepted at AISec 2026; the camera-ready version may differ.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.