How Good Are Time-Series Foundation Models for Pedestrian Crowd Count Forecasting? A Cross-Dataset Comparative Study
Theivaprakasham Hari, Ziteng Li, Yanan Xin, Winnie Daamen, Serge Hoogendoorn
cs.LG · cs.AI
Quick verdict
Two-regime benchmark finds FMs help mainly via long context in seasonal data; solid cost analysis, but untuned baselines and no significance tests weaken it.
Summary
The paper benchmarks seven univariate forecasting methods for pedestrian counts across two contrasting deployment regimes: SAIL2025, a five-day Amsterdam special-event dataset (3-minute resolution, 11 sensors), and Melbourne's hourly sensor network (2010–2017, 16 sensors). The contenders are Seasonal Naive, gradient-boosted trees (LightGBM, CatBoost), deep models (N-HiTS, PatchTST), and zero-shot foundation models (TimesFM, Chronos-2). Under SAIL2025's expanding walk-forward protocol, Seasonal Naive posts the best 24-hour MAE on the high-volume sensor at the final step (89.3, at least 20.2 below every learned model), while Chronos-2 achieves the best zero-shot 2-hour MAE on all three reported sensors at the first step (35.9, 21.5, 11.2). On Melbourne's 2017 test year, long-context inference (TimesFM 16,000 steps; Chronos-2 8,192) cuts TimesFM MAE by roughly 16–49% versus a one-week context, placing FMs first or second at weekly and monthly horizons. A CPU-only cost profile (training time, latency, memory, size) motivates a tiered deployment strategy. Main limitations: fixed untuned baseline configurations, only three representative sensors tabulated per dataset, no significance tests or seed variance, purely univariate inputs without calendar or event covariates, no released code, and a context-length asymmetry favoring FMs over supervised baselines.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- In limited-history event regimes, Seasonal Naive is the strongest long-horizon model on high-volume sensors: E4 24-hour MAE of 89.3 on GASA-03, at least 20.2 points below every learned model (Table III).
- Zero-shot foundation models provide genuine cold-start value that erodes as local data accumulates: Chronos-2 achieves the best 2-hour MAE on all three reported SAIL sensors in E1 (35.9/21.5/11.2) without event data.
- In the seasonal data-rich regime, FM advantage is driven by context length rather than architecture: extending TimesFM context from 168 to 16,000 hours cuts MAE by roughly 16–49% across the three reported Melbourne sensors and horizons (Table IV).
- Deployment cost on identical CPU-only hardware (Table V) supports a tiered strategy: Seasonal Naive for limited-history long horizons, light trees for short horizons, zero-shot FMs at cold start, long-context FMs for planning horizons.
Strengths
Dual-regime design targets a real deployment question
Pairing SAIL2025 (5 days, 3-min, walk-forward E1–E4) with Melbourne (2010–2017 hourly, fixed 2017 test) directly instantiates the two regimes practitioners face — limited-history events vs seasonal data-rich networks — rather than one benchmark distribution (§III-B).
Rare, deployment-grade cost profiling
Table V measures training time, latency (median of 20 origins after warm-up), inference memory, and model size on identical CPU-only hardware: Chronos-2 forecasts in 115–117 ms with a 478 MB checkpoint, TimesFM in ~910 ms with 925 MB, CatBoost needs 781 s to train on Melbourne. Accuracy benchmarks almost never report this.
Quantified zero-shot cold-start value
Honest failure analysis
Internally consistent numbers
Weaknesses
No inferential statistics anywhere
Untuned baselines undermine ranking conclusions
Context-length confound favors FMs
Partial sensor reporting
No code release and profiling protocol inconsistency
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks whether pretrained time-series foundation models (TimesFM, Chronos-2) outperform classical baselines for pedestrian-count forecasting. Seven univariate methods in four paradigms are evaluated on two deliberately contrasting regimes: SAIL2025 (Amsterdam, five days, 3-minute resolution, 11 sensors, expanding walk-forward E1–E4) and Melbourne (16 sensors, hourly 2010–2017, fixed 2017 test year). Central claims: (1) under limited event history, Seasonal Naive is the strongest long-horizon model on high-volume sensors (E4 24-h MAE 89.3, ≥20.2 below all learned models); (2) zero-shot FMs deliver real cold-start value that fades as local data accumulates (Chronos-2 best 2-h MAE on all three sensors in E1: 35.9/21.5/11.2); (3) in the seasonal regime, FM gains are driven by context length rather than architecture; (4) CPU-only cost measurements justify a tiered deployment strategy. The claims are carefully hedged and mostly supported by the extracted evidence, with caveats about baseline tuning and statistical support.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Do time-series foundation models beat simpler methods for pedestrian crowd forecasting?
According to this study, it depends on the regime. With almost no local history (a 5-day event), Seasonal Naive and gradient-boosted trees are competitive and Chronos-2 helps mainly in the first hours: it achieved the best zero-shot 2-hour MAE on all three reported sensors (35.9, 21.5, 11.2). With years of seasonal data (Melbourne), long-context foundation models win: TimesFM with 16,000-step context cut its own MAE by roughly 16–49% versus a one-week context and ranked first or second at weekly and monthly horizons.
What data does the study use and is it public?
Two public sources: SAIL2025 — five days of 3-minute pedestrian counts (Aug 2025) from 11 sensors in Amsterdam's crowd monitoring system, available via the Amsterdam maps portal; and the City of Melbourne open pedestrian counting system — 16 sensors with continuous hourly counts from 2010 to 2017. The paper states no code release, so the pipeline would need to be reimplemented.
Can these forecasting models run on cheap hardware?
Yes — everything was profiled on a laptop-class 13th-gen Intel Core i7-1355U CPU with 16 GB RAM, no GPU. Chronos-2 needed ~115–117 ms and ~875–881 MB per forecast (478 MB checkpoint); TimesFM ~910–913 ms and ~961–973 MB (925 MB checkpoint); Seasonal Naive is essentially free. The slowest to train was CatBoost on Melbourne (781 s, 1.28 GB serving memory), but all refresh cycles fit operational budgets.
What are the main criticisms of the paper?
Baselines used one fixed, untuned configuration with no calendar features, while foundation models received up to 16,000 steps of context versus 168 for baselines — so FM gains partly reflect input information, not just the model. Results are single runs with no significance tests or seed variance, only three representative sensors per dataset are tabulated, and no code is released.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–3 appear only as axis labels and legend text in the extraction; visual claims (e.g., FM over-smoothing in Fig. 2, peak recovery in Fig. 3) cannot be independently verified from the extracted text.
- The extraction may omit an appendix; only 3 representative sensors per dataset are visible in Tables I/III/IV, so full per-sensor results could not be checked.
- No code/data availability statement was found in the extracted text; absence of a release may partly be an extraction artifact.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.