HoneyRoute: Honeypot-Model Routing for Adversarial LLM Serving
cs.CR · cs.CL · cs.LG
Quick verdict
Serving-tier honeypot routing with honest reporting; promising, but headline F1 rests on a 103-request set and hpC fidelity is emulated, not measured.
Summary
HoneyRoute is an inference-serving-layer defense that scores every API request with a streaming router — a frozen ~0.8B embedding backbone plus seven per-domain MLP heads — and diverts requests flagged as malicious to a honeypot rather than the production model; trapped interactions are distilled into attacker fingerprints that retrain the router. Two honeypot implementations are studied: a zero-GPU rule-based responder with selective, camouflaged bait injection, and a trained 9B same-family replica. On a 103-request held-out set the router reaches F1=.911 at 38 ms median added latency, 96% of a two-tier guard-LLM cascade's F1 at 1/385 of its latency; on a 1,103-sample expansion it scores F1=.893 (recall .980, precision .820), below Llama Guard 4's .944. Diverting the malicious share cuts production token consumption 97.8% under concurrent GCG-suffix flooding, with benign p50 latency rising only 21.4->22.7 s. Fidelity judgments (GLM-5.3, cross-checked) give .929 for the replica, .076 for naive bait, .889 for selective bait. Main limitations: headline detection numbers rest on a very small set; the rule-based honeypot's own fidelity is evaluated only via an emulated trap suffix; the retraining loop's gain is a simulated 37-sample correction; attribution precision is weak in absolute terms; benign corpora are largely synthetic or single-site.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A frozen-embedding router with per-domain MLP heads detects malicious LLM requests at F1=.911 (38 ms median added latency), matching 96% of a two-tier guard-LLM cascade's F1 at 1/385 of its latency with 0% evasion on a 13-transformation red-team harness.
- Routing the malicious traffic share to a honeypot cuts production-model token consumption by 97.8% under concurrent GCG-suffix flooding while benign p50 latency rises only from 21.4 s to 22.7 s.
- A trained same-family replica reaches .929 benign-holdout fidelity while naive bait injection collapses to .076 and selective camouflaged injection recovers .889, mapping a fidelity-traceability frontier.
- Trapped-session fingerprints re-link identity-rotated requests to their source campaign at 56.7% top-1 among 30 anchors (17x chance), and a loop-trained correction head cuts misrouting of legitimate security research 9x (FPR .30 to .033) while raising F1 to .933.
- A session-level margin-trend rule detects multi-turn soft-escalation jailbreaks at 100% held-out detection with 8% benign false-flag, where level-based rules false-flag 73-97%.
Strengths
Plausibly novel interception point, carefully positioned
Well-differentiated against protocol-tier honeypots (Sladic et al., VelLMes) and model-internal traps (MemPot, Knowledge Trap) in Sec. 2, with the Bridges et al. (2025) SoK cited as calling for exactly this autonomous serving-tier deception.
Exceptional self-critical reporting
Judge order-swap reveals naive-hpC's .08->.80 position flip yet retains the conservative estimate; expanded-set regression (F1 .911->.893 vs. LG4 .944), 43 benign Chinese FPs, in-sample E7 thresholds later re-validated held-out, and the unvalidated attack-side loop are all stated plainly in E1/E2/E7 and Sec. 5.
Real adversarial payloads and playbooks
Quantified fidelity-traceability frontier with a rate-distortion reading
Cost model tied to measured outcomes
Weaknesses
Headline detection numbers rest on a 103-request set
Rule-based honeypot fidelity never measured intrinsically
Loop gain is a simulated correction, not a demonstrated loop
Attribution is weak in absolute terms, flattered by chance multiples
Benign and multi-turn corpora are synthetic or single-site
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
HoneyRoute (arXiv 2609.08306v3; single independent author) positions itself as the first honeypot deployed at the inference-API serving tier, between gateway and model farm — a gap it locates in the Bridges et al. (2025) SoK taxonomy, since prior LLM honeypots guard SSH/HTTP perimeters and trap-in-model defenses (MemPot, Knowledge Trap) assume the attacker already reached the model. Three claims structure the paper (P1-P3): serving-tier routing of the malicious share to a honeypot; a dual-implementation honeypot with a measurable fidelity-traceability trade-off; and a continuous-analysis loop feeding fingerprints back into detection. Seven experiment blocks (E1-E7, Table 1) support these claims.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is HoneyRoute?
An inference-serving layer that scores each LLM API request with a lightweight detector (frozen 0.8B embedding backbone plus seven per-domain MLP heads) and routes requests flagged as malicious to a honeypot model instead of the production model; trapped interactions are harvested as attacker fingerprints to retrain the detector.
How accurate is HoneyRoute's malicious-request detection?
On a 103-request held-out set it reports F1=.911 at 38 ms median added latency (AUROC .975); on a larger 1,103-sample expansion, F1=.893 with recall .980 and precision .820 — below Llama Guard 4's .944 but at 38-48 ms versus a 14.8 s guard-LLM. Chinese benign questions are a disclosed weak spot.
Can attackers tell they are talking to a honeypot?
The trained 9B replica agrees with production responses on 92.9% of judge-evaluated benign holdout requests. Naive credential-bait injection in every reply is highly visible (7.6% agreement), but selective, camouflaged bait recovers to 88.9% while keeping traceable signals on 83-100% of attacker interactions.
Is HoneyRoute's code and data available?
Partially: the paper states experiment scripts and result JSONs are at github.com/geilihan/honeyroute, with the gateway, both honeypot implementations, and seed corpora described as 'being prepared'. Raw benign production traces cannot be shared; a synthetic substitute is promised.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure graphics (Figs. 1-4) survive extraction only as interleaved axis/legend text; all cited numbers come from captions and body text, which are intact through Appendix A with no end-of-document truncation.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.