SRD-GUARD: A Defense Framework of LLMs via Semantic Rewriting and Joint Multi-Model Scoring for Latent Intent Exposure
Qi Wang, Chengcheng Wan, Jiangtao Wang
cs.CR · cs.AI
Quick verdict
SRD-GUARD couples prompt rewriting with 4-model risk scoring; strong claimed DSR-ORR trade-off, but judging method, stats, and baselines are thin.
Summary
SRD-GUARD is a parameter-free, black-box jailbreak defense that wraps a target LLM with three stages: a rewrite module generates five semantic rewrites of the input that strip contextual packaging while preserving the objective; four independent open-source scorers (Gemma2-9B, Llama3-8B, Mistral-Nemo, Qwen2.5-7B) rate the original and rewrites on a 0-10 risk scale; and a decision module intercepts when any score reaches 6.0, passes through when all fall below 3.0, and otherwise routes gray-zone prompts by the risk change Delta between rewrites and original - sending the highest-risk rewrite with a warning when Delta>0, or the original with a warning. On 300 AdvBench prompts attacked via UNIATTACK, CIPHER, and DeepInception, it reports average DSR of 91.44% on Llama-3-8B-Uncensored and 100% on DeepSeek-V4-Flash, with ORRs of 8% and 12% on 300 OR-Bench-Hard prompts, a better trade-off than DPP, Intention Analysis, and Self-Reminders. Ablations attribute most gains to rewriting (average DSR falls to 50.66%/64.52% without it). Main limitations: the harmfulness/refusal judging procedure is never specified; no significance tests and an unexplained +/- in Table 1; only two targets and three non-adaptive attacks; baselines omit Llama Guard, SmoothLLM, and AutoDefense; and gray-zone routing hands the target a maximally risk-flavored rewrite when Delta>0.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Semantic rewriting that strips contextual packaging is the load-bearing component: removing the Rewrite Module drops average DSR from 91.44% to 50.66% (Llama-3-8B-Uncensored) and from 100% to 64.52% (DeepSeek-V4-Flash) (Sec 5.4, Table 1).
- Joint scoring by four independent LLM scorers prevents single-evaluator over-aggression: a single Llama3-8B scorer raises ORR to 37%/50% while the full ensemble keeps it at 8%/12% (Sec 5.4).
- The relative-risk decision rule yields a more favorable DSR-ORR trade-off than DPP, IA, and SR, improving average DSR by 30.30 and 15.67 percentage points while reducing ORR by 10.04 and 10.22 percentage points (Sec 5.2-5.3).
- SRD-GUARD attains 100% DSR across all three attacks on the safety-aligned DeepSeek-V4-Flash with only 12% ORR, by intercepting only scores of 6.0+ and warning-routing the gray zone (Sec 5.2).
Strengths
Novel relative-risk signal
The decision module's Delta = Svar - S0 (Sec 4.4) uses the risk change under de-contextualizing rewrites as evidence of concealment, and gray-zone requests are routed rather than uniformly rejected - a mechanism distinct from purely threshold-based filters.
Trade-off evaluated on both axes
Over-refusal is measured on 300 OR-Bench-Hard prompts alongside DSR (Sec 5.2-5.3), making the DSR-ORR trade-off explicit: 91.44%/8% on Llama-3-8B-Uncensored and 100%/12% on DeepSeek-V4-Flash.
Thorough ablation with honest anomaly reporting
Sensitivity analysis includes negative results
Deployability framing without overstatement
Weaknesses
Judging methodology unspecified
No statistical inference
Thin baseline field despite broader related work
Unquantified harm-amplification pathway
Limited external validity
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
SRD-GUARD (arXiv 2609.06540) is an inference-time, black-box jailbreak defense with three stages: a Rewrite Module (five objective-preserving rewrites via deepseek-v4-flash-thinking), a Scoring Module (Gemma2-9B, Llama3-8B, Mistral-Nemo, Qwen2.5-7B rating the original and rewrites 0-10, Eqs. 1-3), and a Decision Module (Sec 4.4): hard-intercept at any score >= 6.0, pass-through when all < 3.0, and gray-zone routing by Delta = Svar - S0 - forwarding the highest-risk rewrite with a warning when Delta>0, otherwise the original with a warning. Headline claims: average DSR 91.44% (Llama-3-8B-Uncensored) and 100% (DeepSeek-V4-Flash) against UNIATTACK, CIPHER, and DeepInception on 300 AdvBench prompts; ORR 8%/12% on 300 OR-Bench-Hard prompts; a more favorable DSR-ORR trade-off than DPP, IA, and SR; rewriting as the load-bearing component.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is SRD-GUARD and how does it defend against LLM jailbreak attacks?
SRD-GUARD is an external, parameter-free defense layer placed between the user and a target LLM. It rewrites each input into five versions that strip away role-playing or scenario packaging, has four independent open-source LLMs score all versions for risk on a 0-10 scale, and then either intercepts the request (any score of 6.0 or higher), passes it through unchanged (all scores below 3.0), or, for gray-zone cases, routes the version whose risk rose most after rewriting - with a safety warning - to the target model.
How well does SRD-GUARD perform?
The paper reports an average Defense Success Rate of 91.44% against three jailbreak attacks (UNIATTACK, CIPHER, DeepInception) on Llama-3-8B-Uncensored and 100% on DeepSeek-V4-Flash, with over-refusal rates of 8% and 12% on 300 OR-Bench-Hard benign prompts. Ablations attribute the largest share of protection to the rewriting step, which the authors say exposes intent hidden by contextual packaging.
What are the main limitations of SRD-GUARD?
The paper does not specify how harmful outputs and refusals were judged; it reports no statistical significance tests and leaves the +/- values in its ablation table undefined; it evaluates only two target models against three non-adaptive attacks on 300-prompt samples; strong external defenses like Llama Guard are discussed but not compared; and its gray-zone rule forwards the highest-risk rewrite to the target model, which could amplify harm if the risk scores are wrong.
Is code available for SRD-GUARD?
The abstract points to an anonymous artifact repository (anonymous.4open.science). However, the paper text itself does not include the rewriting or scoring prompts or the metric-judging procedure, so independent reproduction would depend on what those artifacts actually contain.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 3's per-attack values are cited in this review only from numbers quoted in the prose; the figure graphics themselves are not recoverable from the extracted text.
- The provided text ends after the references; any appendix content beyond that point is unverifiable from this extraction.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.