Understanding Dynamic Scenes at Gigapixel Scale: Wide-Area Spatio-Temporal Perception from UAVs
Yuhang Zhu, Meiyi Zhu, Yunkai Dang, Zhangnan Li, Yuxuan Wang, Wenbin Li, Hongbing Pan
cs.CV
Quick verdict
HARD: 122 MP temporal UAV benchmark with s-HOTA latency metric; valuable contribution, but VQA co-built by evaluated GPT-5.5 and streaming eval is Car-only.
Summary
The paper introduces HARD, a UAV benchmark of 3,549 frames at a fixed 12768x9564 (122 MP) resolution, captured over 21 sequences at 50-355 m altitude in four flight modes and annotated at three levels: object detection (138,830 boxes, four classes), multi-object tracking (per-instance cross-frame identities), and scene-level VQA (1,565 expert-verified QA pairs over eight types). Because one frame takes seconds to process, the authors propose streaming-HOTA (s-HOTA), which scores each ground-truth instant against the freshest output available under a measured latency profile (frame interval P = 2 s), extending streaming perception to tracking. Baselines cover 20 detectors (best mAP50:95 = 0.305, Cascade R-CNN), 80 detector-tracker pipelines, and 12 zero-shot VLMs. Reported findings: the tiling pipeline shapes detection accuracy and latency as much as the detector (tile overlap 0 to 150 px raises mean mAP 0.166 to 0.213 while latency grows 3.31 to 5.90 s); streaming latency penalizes association far more unevenly than detection, and offline leads of up to 16.8 HOTA points compress to near-ties or reversals under s-HOTA; VLM failures split into single-frame perception (29.2-66.5% spread) versus near-chance cross-frame identity binding, where only GPT-5.5 exceeds chance (66.3% on Relocate). Main limitations: streaming evaluation restricted to the Car class, no statistical uncertainty reported, and VQA questions co-constructed with GPT-5.5, which is also evaluated.
Review complete
Assessment at a glance
6
risk flags identified
● high 1
● medium 4
● low 1
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- HARD is the first benchmark to jointly provide UAV-platform imagery, ultra-high (122 MP) resolution, temporal continuity, multi-class tracking, and scene-level VQA annotation.
- At gigapixel scale, the end-to-end tiling pipeline affects detection accuracy and latency as much as the detector choice itself (Finding 1, Table 2).
- The proposed streaming-HOTA charges end-to-end latency: streaming degrades the association axis three times more unevenly than the detection axis, and offline leads of up to 16.8 points compress to near-ties or reversals, with the winner governed by the load ratio rho = latency / frame interval.
- Zero-shot VLM failures separate into a single-frame perception axis (37-point spread) and a cross-frame identity-binding axis where eleven of twelve models stay near chance, and progress on one axis does not transfer to the other.
Strengths
Fills a demonstrable dataset gap
Figure 1 situates HARD against VisDrone/UAVDT (about 0.55-3 MP), PANDA (gigapixel but ground-based, slanted view), and single-image remote-sensing VQA sets; HARD is uniquely UAV platform + 122 MP + temporal continuity + multi-class tracking + scene-level VQA, backed by a real capture campaign (Phase One iXM-GS120, 21 sequences, 50-355 m, four flight modes producing translation, rotation, parallax, and static-background conditions).
s-HOTA is a principled and timely metric
Section 4 extends streaming perception (Li, Wang, and Ramanan 2020) from detection AP to HOTA with an explicit availability policy (Eq. 1-3) and a K-worker no-queue deployment; Table 3 shows the practical consequence, e.g., Cascade R-CNN + StrongSORT loses 32.6 points (54.5 to 21.9) and Faster R-CNN rankings reverse (StrongSORT 17.7 vs ByteTrack 19.4 under s-HOTA), results offline HOTA cannot express.
Unusually broad baselines with actionable systems analysis
Candid self-limitation increases trust
Annotation rigor and ethics statement
Weaknesses
Streaming analysis restricted to one class
No uncertainty quantification
VQA construction involves an evaluated model
Tracker comparison carries configuration confounds
Identity-binding claim not disentangled from input downsampling
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the HARD dataset and how large is it?
HARD (ultra-High-resolution Airborne Remote-sensing Dataset) is a UAV benchmark of 3,549 frames captured at a fixed 12768x9564 (122 MP) resolution in 21 sequences from 50-355 m altitude and four flight modes. It contains 138,830 bounding boxes in four classes (pedestrian, car, bus, truck) with cross-frame identities for multi-object tracking, plus 1,565 expert-verified scene-level question-answer pairs spanning eight question types. Five of the 21 sequences are held out as the test set.
What is streaming-HOTA (s-HOTA)?
It is a latency-aware version of the HOTA multi-object-tracking metric. Because one 122 MP frame takes seconds to process, a tracker's output is stale by the time it exists. s-HOTA pairs each ground-truth moment with the most recent output available at that moment, given a per-frame latency profile and frame interval (P = 2 s in the paper), then computes standard HOTA on those latency-adjusted pairs. Pipelines are thus charged for staleness, and rankings that hold offline can compress or reverse under it.
How is HARD different from PANDA or VisDrone?
VisDrone and UAVDT offer UAV video but only about 0.55-3 MP frames, too coarse to keep every target identifiable over a wide area. PANDA offers gigapixel resolution and temporal sequences but from fixed ground cameras looking outward at a slant, not a UAV's near-vertical geometry, and without scene-level VQA. HARD is the first benchmark combining a UAV platform, 122 MP frames, temporal continuity, multi-class tracking, and scene-level VQA in one corpus.
How well do vision-language models perform on HARD's VQA benchmark?
Twelve models evaluated zero-shot split into two failure modes. On single-frame perception questions, accuracies spread widely, from 29.2% (GPT-4o) to 66.5% (Gemini 3.1 Pro). On cross-frame identity questions such as Relocate, eleven of twelve models stay in a narrow band near chance (temporal-axis range 35.6-49.0%); only GPT-5.5 clearly exceeds chance, at 66.3% on Relocate. The authors read this as general capability helping perception but not identity binding across frames.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures are available only as caption text; per-type VQA chance levels, the Figure 4b pipeline annotations, and the Pareto-frontier detail in Figure 3b could not be visually verified.
- Supplementary material (viewing geometry per flight mode, s-HOTA deployment parameters, QA generation procedure, CMC ablation) is referenced throughout but not included in the provided text.
- No explicit data/code release URL or license statement appears in the extracted main text; codeAvailable is set false on that absence rather than an explicit refusal to release.
- arXiv v1 preprint (September 2026), apparently not yet peer-reviewed; table text was extracted with rows run together, so individual cell values were cross-checked against the surrounding prose where possible.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.