LLM-Generated Feature Pools for Time Series Anomaly Detection
Youssef Attia El Hili, Malik Tiomoko, Corinne Ancourt
cs.AI
Quick verdict
Claims a 10-feature MAD pipeline beats neural/statistical baselines on TSB-AD-U; LLM-generated pools lift it to 0.588, matching best pretrained. Convincing.
Summary
The paper asks what limits feature-based anomaly detection on univariate time series: the detector, its tuning, or the feature vocabulary. The authors build a deliberately minimal pipeline — sliding windows at each series' estimated dominant period, ten hand-crafted window statistics, a transductive median/MAD robust z-score, and per-domain feature selection on a held-out tuning split — and evaluate on TSB-AD-U (48 tuning, 350 evaluation series, nine domains, VUS-PR). The hand-crafted pipeline scores 0.529 per-series VUS-PR, above the best neural (0.45, StreamVAE) and statistical (0.44, MMPAD) leaderboard entries and 0.06 below the best pretrained foundation model (Time-RCD+MAFT, 0.59). Ablations (Figure 4, Tables 2, 3) attribute performance to the candidate pool: selection strategies span 0.031, the aggregation grid 0.096, while changing the pool spans 0.226; the larger static library catch22 is weakest at 0.362. The authors then prompt three multimodal LLMs with ten rendered example windows per domain to write ten NumPy feature functions; the best generated pool (gemini-3.5-flash) alone reaches 0.569, and selecting over its union with the hand-crafted features reaches 0.588, beating the generated pool in all twelve generator-seed pairs (exact sign test, p=4.9e-4). Main limitations: a single univariate benchmark, baselines taken as published rather than re-tuned, very small domain splits (traffic has five evaluation series), and only three generators times four seeds with single-round generation.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A ten-feature transductive median/MAD pipeline with per-domain feature selection reaches 0.529 per-series VUS-PR on TSB-AD-U, above the best published non-pretrained entries (StreamVAE 0.45, MMPAD 0.44) and 0.06 below the best pretrained entry (Time-RCD+MAFT 0.59).
- Across selection strategies (including a hindsight oracle), aggregation settings, and candidate pools, the pool is the binding constraint (0.226 range vs 0.031 and 0.096), and enlarging the static library catch22 (0.362) moves the score in the wrong direction.
- Multimodal LLMs prompted with rendered example windows produce per-domain feature pools competitive with expert design under matched selection (best: gemini-3.5-flash at 0.569 alone).
- Selecting over the union of hand-crafted and generated features beats the generated pool alone in all twelve generator-seed pairs (exact sign test, p=4.9e-4) and lifts the pipeline to 0.588, matching the best pretrained leaderboard entry.
Strengths
Strict, fully documented selection protocol
Labels touch only per-domain feature selection on the 48-series tuning split; the 350 evaluation series are scored once after all design choices are frozen (Sec. 3.4, Sec. 4, App. A/B, Table 7). Window length is fixed a priori to the unsupervised period estimate, leaving only the two aggregation choices and the feature subset as searched quantities.
Effect-size attribution is unusually clean
Figure 4 and Tables 2, 3, 10, 13 isolate selection (0.031 range including a hindsight oracle at 0.551), aggregation (0.096 full grid, only 0.021 among non-median feature aggregations under mean point aggregation) versus candidate pool (0.226, from catch22 0.362 to union 0.588), directly supporting the central pool-is-the-ceiling claim.
Exactly paired evidence for the LLM complementarity claim
Full disclosure of the generative apparatus
Honest scope discipline
Weaknesses
Baselines are never re-run or re-tuned
Single benchmark, univariate track only
Small-domain statistics are fragile
Generation variance is narrowly sampled
No code or artifact release stated
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper studies univariate time series anomaly detection on TSB-AD-U and makes four claims: (1) a deliberately minimal feature-based pipeline — ten hand-crafted window statistics scored by a transductive median/MAD rule with per-domain feature selection on the tuning split — reaches 0.529 per-series VUS-PR, above the best published non-pretrained entries (StreamVAE 0.45, MMPAD 0.44) and 0.06 below the best pretrained foundation model (Time-RCD+MAFT, 0.59; Table 1); (2) ablations show the candidate feature pool, not selection strategy (0.031 range including a hindsight oracle) or aggregation (0.096), is the binding constraint (Figure 4); (3) per-domain feature pools written by multimodal LLMs from rendered example windows are competitive alone under matched selection (best: gemini-3.5-flash at 0.569, Table 4); (4) selecting over the union with the hand-crafted features wins all twelve generator-seed pairs (exact sign test, p=4.9e-4, Table 12) and reaches 0.588, matching the leaderboard's best entry. The scope is deliberately narrow: one benchmark, one track, three generators, four seeds, single-round generation.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is VUS-PR and why is it used for time series anomaly detection?
VUS-PR (Volume Under the PR Surface) is a threshold-free accuracy metric that integrates the precision-recall curve over a range of buffer widths around each labelled anomalous region, so it tolerates small localisation errors. The TSB-AD benchmark sets the buffer to each series' estimated dominant period and reports the mean VUS-PR over all series as its headline number.
How does the anomaly detection pipeline in this paper work?
Each time series is cut into sliding windows spanning its estimated dominant period; ten simple statistics (e.g. standard deviation, skewness, autocorrelation) are computed per window; each feature is converted to a robust z-score using a median and MAD fitted on the same series; scores are averaged across features and across overlapping windows into a per-point anomaly score. A small feature subset per domain is chosen using labels from a separate tuning split, and the evaluation series are scored once with the frozen configuration.
Does this method beat deep learning or foundation models on TSB-AD-U?
It beats the best published non-pretrained entries: 0.529 per-series VUS-PR versus 0.45 (StreamVAE, best neural) and 0.44 (MMPAD, best statistical). Against pretrained foundation models it is competitive rather than superior: with an LLM-generated feature pool unioned with the hand-crafted one it reaches 0.588, essentially matching the leaderboard's best entry (Time-RCD + MAFT at 0.59) but not exceeding it.
Is code or data available for this paper?
The paper text contains no code or repository link. The TSB-AD-U benchmark itself is public and the appendices give the full generation prompt, hyperparameters, per-seed results, and API costs, so the deterministic parts of the pipeline could be re-implemented; reproducing the LLM-generated pools requires API access to the specific commercial models used (gemini-3.5-flash, gemini-3.1-flash-lite, claude-sonnet-5).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-4 are available only through captions and quoted values; bar spans and visual details could not be inspected directly.
- Baseline values are quoted by the paper from the public TSB-AD-U leaderboard and were not independently verified against the leaderboard itself.
- The LLM results depend on versioned commercial APIs whose behaviour may change, so generated-pool numbers may not reproduce exactly even with the full prompt.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.