MoveBench: A Benchmark for Global-Scale Wildlife Movement Forecasting
Justin Kay, Shir Bar, Ellen O. Aikens, Martin Becker, Francesca Cagnacci, Juliet Cohen, Scott W. Forrest, Jessica Kendall-Bar, et al.
cs.LG
Quick verdict
A well-designed first benchmark for probabilistic wildlife movement forecasting, but withheld data/code leaves headline findings unverifiable.
Summary
The paper introduces MoveBench, presented as the first large-scale benchmark for probabilistic wildlife movement forecasting. It aggregates 2.6 million curated GPS locations from over 800 individuals across 110 species and 127 countries, drawn from 147 datasets (Movebank Data Repository, a literature review, and expert nominations). Each location is paired with rasters for 160 environmental covariates - weather, vegetation, DEM, marine, human footprint, and AlphaEarth embeddings - preprocessed into 1.6 billion 101x101 tiles at four spatial resolutions. The authors formalize forecasting at horizons of 1 hour to 7 days, argue that point metrics like L2 reward degenerate 'don't move' predictions, and evaluate with the energy score, a proper scoring rule. Two hold-out splits (temporal and individual) probe distinct generalization axes. Comparing correlated random walks, linear iSSF, deepSSF, and MoveFormer, they find methods generalize better to future timepoints than to unseen individuals, deep learning does not consistently beat simple baselines, and covariate choice matters; deepSSF is best in-distribution but worst out-of-sample, while MoveFormer achieves the best average. Limitations: data and code are withheld until acceptance, some studies have single individuals or unspecified licenses, and visible reporting lacks uncertainty quantification.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
4 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- MoveBench is the first large-scale benchmark for probabilistic wildlife movement forecasting, with 2.6M GPS fixes, 800+ individuals, 110 species, 127 countries, and 160 covariates as 1.6B raster tiles.
- Point-prediction metrics such as L2 are inadequate for stochastic movement because they reward degenerate central predictions; the multivariate energy score enables proper evaluation of any forecasting model.
- Existing methods generalize better to future timepoints than to unseen individuals, and most do not consistently beat a correlated-random-walk baseline on held-out individuals.
- Deep learning approaches (deepSSF, MoveFormer) do not consistently outperform a linear iSSF; deepSSF's in-distribution advantage reverses out-of-sample while MoveFormer achieves the best average.
- Environmental covariate selection significantly impacts forecasting performance.
Strengths
Unmatched scale and ecological breadth for the task
2.6M curated fixes from 800+ individuals, 110 species, 127 countries, aggregated from 147 datasets with per-study license and fix-interval documentation (Table 3); Table 1 shows orders-of-magnitude wider spatial (283-3500 km IQR) and temporal (818-8000 h IQR) ranges than GeoLife, Porto, T-Drive, or WorldTrace.
Principled metric redesign grounded in proper scoring rules
Sec. 5 motivates the energy score (multivariate, density-free, proper; refs Gneiting et al.) and Fig. 2 provides an empirical demonstration on real MoveBench data that expected L2 is minimized by a degenerate always-predict-don't-move forecaster, making the protocol critique concrete rather than rhetorical.
Dual hold-out design isolates two distinct generalization failures
Covariate infrastructure at unprecedented scale
Weaknesses
Core artifact not released at review time
Confounded comparison behind the winning model
Degenerate cases in the study pool unhandled visibly
No visible uncertainty quantification
License gaps in included data
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
MoveBench (arXiv 2609.15780) is a benchmark paper with three deliverables: (1) a curated dataset - 2.6M GPS fixes from 800+ individuals, 110 species, 127 countries, aggregated from 147 source datasets (Movebank Data Repository, a literature review, and MoveBON expert survey; Sec. 4, Table 3); (2) an environmental context stack - 160 covariate layers (weather/climate 51, vegetation/land cover 34, AlphaEarth embeddings 64, DEM 4, marine 6, human footprint 1, plus movement/time features; Table 2) preprocessed into 101x101 rasters at four resolutions, 10.4M stacks totaling 1.6B layers; and (3) an evaluation protocol for probabilistic forecasting at 1 h, 12 h, 1 day, and 7 day horizons, scored with the multivariate energy score (Sec. 5). Central claims: L2-style point metrics are inadequate for stochastic movement (Fig. 2b shows the best expected-L2 forecast is 'always predict don't move'); methods generalize better to future timepoints than unseen individuals; deep models do not consistently beat simple baselines; covariate selection materially changes performance. The claims are internally consistent with the presented evidence, but the artifact itself is not yet released (footnote 2: release 'following acceptance'), so external verification is currently impossible.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is MoveBench?
MoveBench is a benchmark dataset and evaluation protocol for probabilistic wildlife movement forecasting. It contains 2.6 million curated GPS locations from over 800 individuals across 110 species in 127 countries, paired with 160 environmental covariates preprocessed into 1.6 billion raster tiles, plus code for training and evaluating forecasting models. A smaller MOVEBENCH-Mini subset of ten studies supports quick experiments.
Why does the paper reject average displacement error (L2) for animal movement?
Because animal movement is stochastic and often multimodal, a single best-guess point is a poor target: L2 is minimized by predicting a central location, so a model that always predicts the animal stays where it is can score best while being useless. MoveBench instead scores full predictive distributions with the energy score, a proper scoring rule that rewards calibrated, sharp probabilistic forecasts from any model.
What are the main findings of the MoveBench experiments?
Across four method families (correlated random walk, linear iSSF, deepSSF, MoveFormer), models generalized better to future timepoints of known individuals than to unseen individuals; most methods failed to beat a simple correlated random walk on unseen individuals; deep learning was not consistently better than linear baselines; and the choice of environmental covariates significantly affected performance.
Is MoveBench data publicly available?
According to the preprint, the benchmark, evaluation code, and the raster-preparation pipeline will be released publicly after the paper is accepted. Most underlying GPS datasets come from the Movebank Data Repository under Creative Commons licenses, though a few included studies currently have unspecified licenses.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Sections 5 (energy score definition) and 6 (results tables and numbers) were partially omitted in the text extraction; judgments about statistical reporting rest on visible text only.
- Figures 1, 2, and 4 could not be analyzed; quantitative statements are taken from captions and body text.
- Appendix B (curation criteria, split construction details, full covariate list) was not included in the extracted text.
- The preprint is marked 'under review' with artifact release deferred to acceptance; availability claims may change.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.