Comparative Evaluation of Static Embedding Models for HTTP Request Anomaly Detection
Amanda Riverol, Gustavo Betarte, Rodrigo Martínez, Álvaro Pardo
cs.CR · cs.AI
Quick verdict
Benchmarks static embeddings plus one-class detectors for HTTP anomaly detection across three datasets, showing strong FastText consistency and coverage; but internal inconsistencies, no significance testing.
Summary
This paper benchmarks static embedding models (Word2Vec, FastText, Doc2Vec) inside a single modular pipeline, HEDA, that combines embeddings with one-class detectors (OCSVM, LOF, Isolation Forest) trained exclusively on benign HTTP traffic. Evaluation uses three datasets of differing character: Drupal (self-derived), synthetic CSIC 2010, and honeypot-collected SR-BH 2020. Operating points are chosen without labels via the Lee–Liu F-hat measure on a validation set. FastText gives the most consistent results: the best Drupal configuration (d=100, w=7) achieves AUC 0.96, TPR 80.11% at FPR 0.79%, far ahead of ModSecurity CRS v3 (TPR 29.55%/77.89% at FPR 15.57%/49.93%), and on SR-BH 2020 it beats the authors' prior work (90.09% TPR, 3.43% FPR vs 78.87%/0.84%). On CSIC 2010 detection drops (TPR 62.99% for the headline model, while LOF on the same embedding reaches 86.94%). The advantage is plausibly explained by subword coverage: FastText represents 100% of test tokens versus 99% for alternatives, and 100% of attack requests versus 6.6%. Key limitations: no variance or significance testing despite many comparisons; the stated grid (36 configurations, 12 evaluated) contradicts reported tables (d=50 absent); an unexplained FastText-OCSVM FPR discrepancy on CSIC between Tables VI (1.29%) and VII (3.07%); an over-abstract claim; missing TF-IDF baselines under identical tokenization; and no efficiency measurements supporting the real-time argument. No code release is stated.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- The choice of static embedding significantly affects one-class HTTP anomaly detection, and FastText yields the most consistent results across the three evaluated datasets.
- The best Drupal configuration (FastText, d=100, w=7, OCSVM) achieves AUC 0.96, TPR 80.11% at FPR 0.79%, far surpassing ModSecurity OWASP CRS v3 (PL1: TPR 29.55% at FPR 15.57%).
- FastText's subword modeling eliminates the OOV problem — 100% token and request coverage (vs 6.6% fully-covered attack requests for Word2Vec/Doc2Vec) — explaining its robustness.
- Among one-class detectors, OCSVM shows the most stable behavior in combination with static embeddings, and FastText-OCSVM performs comparably to or better than prior expert-feature methods at much lower FPR (e.g., 0.79% vs 2.29-6.00% on Drupal).
Strengths
Controlled apples-to-apples benchmark design
HEDA (Fig. 3) fixes preprocessing (double URL-decode, lowercase, NLTK tokenization), mean pooling, and the detector while swapping only the embedding, directly supporting the paper's causal claim that representation choice drives performance.
Honest unsupervised protocol
Both embeddings and detectors are trained only on benign traffic, and thresholds are calibrated with the label-free F-hat measure of Lee & Liu (Section V) — no test-attack leakage in model selection, unlike many comparable studies.
Mechanistic explanation of FastText's advantage, backed by a coverage table
Heterogeneous, security-relevant evaluation
Large hyperparameter sweep
Weaknesses
Internal inconsistencies in the experimental record
No statistical rigor
Headline claim oversold on CSIC 2010
Baseline design cannot support the automation claim
Efficiency claim unsupported
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Preliminary Review: Comparative Evaluation of Static Embedding Models for HTTP Request Anomaly Detection
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is HEDA and how does it detect malicious HTTP requests?
HEDA (HTTP Embedding-based Detection Architecture) is a pipeline that turns each HTTP request into text, tokenizes it, embeds tokens with a static embedding model, averages the vectors into one request vector, and feeds it to a one-class anomaly detector. Both the embedding and the detector are trained only on benign traffic; requests whose anomaly score exceeds a calibrated threshold are flagged as attacks.
Which embedding works best for HTTP request anomaly detection according to this paper?
FastText. Its character n-gram subword modeling represents out-of-vocabulary tokens, covering 100% of test tokens versus 99% for Word2Vec and Doc2Vec, and it achieved the most consistent detection across the Drupal, CSIC 2010, and SR-BH 2020 datasets. The best Drupal configuration reached AUC 0.96, a true positive rate of 80.11%, and a false positive rate of 0.79% with a one-class SVM.
Why does FastText outperform Word2Vec on HTTP attack detection?
Malicious requests are full of rare or obfuscated tokens never seen in benign training traffic. Word2Vec cannot represent such out-of-vocabulary tokens — in this paper only 6.6% of attack requests were fully representable — while FastText builds vectors from character n-grams, achieving 100% coverage of attack requests. Fewer invisible tokens means request vectors better separate attacks from benign traffic.
How does FastText-OCSVM compare to ModSecurity WAF rules (OWASP CRS)?
On the paper's Drupal benchmark, ModSecurity with OWASP CRS v3 at paranoia level 1 detected 29.55% of attacks with a 15.57% false positive rate, and at level 2 detected 77.89% with 49.93% false positives. FastText-OCSVM detected 80.11% with only 0.79% false positives — comparable detection to PL2 with roughly 60x fewer false alarms, though this comparison holds for the paper's specific evaluation setup.
Does the proposed method work well on the CSIC 2010 dataset?
Less well than elsewhere. The headline FastText-OCSVM model detected only about 63% of attacks (AUC 0.79) on CSIC 2010, which the authors attribute to the dataset's synthetic nature; a Local Outlier Factor detector on the same FastText embedding did better (86.94% TPR, 0.96 AUC). The paper's own tables show the best detector pairing is dataset-dependent.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-5 are unavailable as images; t-SNE projections, ROC curves, and score-distribution plots (Fig. 5) could not be assessed, and AUC claims rest on reported table values only.
- The extracted text is plain ASCII; malformed author names and math/symbols (e.g., F-hat) are extraction artifacts, and the '22 Sep 2026' arXiv stamp may be an extraction error.
- The SR-BH 2020 evaluation protocol (binary aggregation vs per-attack-category) is not fully specified in the text; Table VI/VII figures could not be cross-checked against the original PDF layout.
- Reference [18]'s bibliographic details are incomplete in the source text, so the Table II attribution mismatch ('Khalil et al.' vs Gniewkowski et al.) could partly stem from extraction corruption.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.