HiveTraceGuard-Pro: A Compact Generative Guardrail for Prompt Injection, Jailbreaks, and Adversarial Obfuscation
Nikita Oblakov, Sabrina Sadiekh, Evgeniy Kokuykin
cs.CR · cs.AI
Quick verdict
0.6B RU/EN guardrail claims top Russian robustness and lowest latency; transparent reporting, but headline wins rest on self-authored, contaminated sets.
Summary
The paper presents HiveTraceGuard-Pro, a 0.6B generative guardrail LoRA-tuned from Qwen3-0.6B for binary safe/unsafe classification of user requests and assistant replies, trained on a 464,092-row Russian- and English-language corpus with paired benign/harmful examples and eight obfuscation transforms applied to both labels. In a single harness the authors score thirty-five guards on nineteen benchmark groups: the model's aggregate key is 0.7432, behind YuFeng-XGuard-Reason-8B (0.7641) and SingGuard-2B (0.7552), and 0.7153 over the sixteen public groups, where four guards score higher. Among fifteen models it posts the highest clean Russian robustness combined-F1 (0.88) and Russian prompt-injection recall (0.999) alongside the run's lowest median latency (14.3 ms). However, both headline Russian results use self-authored sets, and at least 27.1% of the prompt-injection set overlaps the training corpus. All response results use a legacy standalone-reply serialization rather than the deployment chat-template path. Pooled FPR is 0.268 against a field median of 0.227, and the reserved confirmation set shows response FPR 0.563. Weights are released under Apache-2.0; the corpus, evaluation sets and evaluation code remain internal.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A 0.6B generative guardrail LoRA-tuned from Qwen3-0.6B reaches an aggregate key of 0.7432 across nineteen benchmark groups, behind only an 8B guard (0.7641) and a 2B guard (0.7552), and ahead of every equal-or-smaller guard in the suite.
- On the authors' Russian evaluation sets it achieves the highest clean robustness combined-F1 (0.88) and Russian prompt-injection recall (0.999) among fifteen guards, with the lowest median latency (14.3 ms) in that run.
- Domain-paired benign/harmful training examples with eight obfuscation transforms applied to both labels yield harm-F1 0.852 on augmented requests while making over-blocking measurable per topic.
- Restricted to the sixteen public benchmark groups, the model's key drops to 0.7153 with four of the thirty-four other suite guards above it.
- The FPR/FNR trade-off under obfuscation persisted across six additive data rounds and an objective change, and cannot be attributed to data versus capacity because all experiments keep 0.6B.
Strengths
Unusually candid self-auditing
Section A.4 quantifies contamination and rescoring: Aegis-2.0 prompt F1 moves 0.8229 to 0.8230 after removing 199 exact matches (0.809 at a 50% cut); S-Eval recall drops 0.806 to 0.784 after dropping the contaminated category; Section 4.1 openly reconciles the two harnesses cell by cell.
Single-harness suite with reruns, not copied numbers
35 guards scored over 19 groups from 44 datasets via per-model adapters preserving each model's published prompt and decode format (Tables 2 and 11); Section D documents per-model row shortfalls and shows the aggregation favors, not penalizes, competitors on parse failures.
Compact-model Pareto position is real in this run
Reserved confirmation set with honest negative result
Artifact-level reproducibility of the model itself
Weaknesses
Headline Russian wins rest on self-authored, partially contaminated sets
All response-side numbers use a non-deployment serialization
Rankings lack uncertainty support
Over-blocking is the dominant error mode and resists mitigation
Corpus governance gaps
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
HiveTraceGuard-Pro is a 0.6B generative guardrail (Qwen3-0.6B + LoRA) emitting one binary safe/unsafe verdict for the final user turn or assistant reply, trained on a 464,092-row Russian/English corpus with topic-paired benign/harmful examples and eight obfuscation transforms applied to both labels. Central claims: (1) aggregate key 0.7432 across 19 groups, behind only YuFeng-XGuard-Reason-8B (0.7641) and SingGuard-2B (0.7552) and ahead of every equal-or-smaller guard; (2) highest clean Russian combined-F1 (0.88) and PI-RU recall (0.999) in a fifteen-model cohort; (3) lowest median latency, 14.3 ms; (4) public-groups-only key of 0.7153 with four guards above. The abstract and conclusion are unusually careful, disclosing the 27.1% overlap, the self-authored provenance of the Russian sets, and the legacy response serialization up front.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is HiveTraceGuard-Pro?
An open-weights (Apache-2.0) 0.6B-parameter LLM safety guardrail fine-tuned from Qwen3-0.6B with LoRA. It reads a dialogue and outputs a single binary verdict (safe/unsafe) for the final user request or assistant reply, with training and evaluation focused on Russian and English prompt injection, jailbreaks, and obfuscation attacks.
How does HiveTraceGuard-Pro compare to Llama Guard or Qwen3Guard?
In the authors' 19-group harness its aggregate score is 0.7432, behind YuFeng-XGuard-Reason-8B (0.7641) and SingGuard-2B (0.7552) but ahead of all equal-or-smaller guards, including Qwen3Guard-Gen-0.6B (0.7118). On the sixteen public groups only, its 0.7153 is beaten by four guards. It is much faster: 14.3 ms median latency in their benchmark run.
Is the 0.999 Russian prompt-injection recall trustworthy?
It was measured on the authors' own attack-only set, of which at least 27.1% overlaps the training corpus, so it should not be read as an independent generalization estimate — the paper itself states this. The set also contains no benign requests, so it cannot measure over-blocking.
Can I run HiveTraceGuard-Pro in production?
Yes; the merged weights, exact inference contract (verdict token ids 18675/38157, bundled chat template, greedy one-token decode) and a reproduction snippet are public. Caveats: pooled FPR is 0.268, a probe showed a harmful request buried under ~80,000 filler tokens was labeled safe (so enforce a context cap preserving both prompt ends), and all published response-side numbers use a legacy serialization rather than the natural assistant-role path.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure-embedded data (Figures 2-4) cannot be fully verified from text extraction; all values cited in this review come from tables and body text, which extracted cleanly.
- The metadata abstract is cut mid-sentence ('...serialization rather'), but the complete sentence appears in the paper body, so no content is lost.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.