Optical-Flow Wingbeat Counting in MuJoCo: A Comparison of Convolutional, Spiking, and Attention-Based Temporal Models
cs.CV
Quick verdict
Controlled MuJoCo study: causal TCN, spiking and attention models reach 92-97% exact wingbeat counts; rigorous but simulation-only, one seed, no ranking.
Summary
This paper presents a controlled simulation study of wingbeat counting from video. In MuJoCo, three flapping-wing targets adapted from open-source projects were observed by virtual Crazyflie-mounted cameras at 1.5 m and 3.0 m, yielding 1,440 clips from 240 paired scene configurations with a scene-level 3:1 train-test split. Signed Farneback optical flow fed a shared CNN spatial encoder followed by one of three causal temporal modules - a dilated TCN, a recurrent LIF spiking network, or ALiBi causal self-attention - all predicting phase and activity, decoded by an identical directed-crossing counter with hysteresis. Exact-count accuracy ranged from 95.00-96.67% at 1.5 m and 92.22-95.00% at 3.0 m. Paired scene-cluster bootstrap intervals for all accuracy differences included zero; far-distance event-F1 differences did exclude zero against the spiking model. Post-hoc analysis showed most errors cluster within one camera frame of evaluation boundaries, and seven far-distance spiking-model clips had correct totals despite event-timing mismatches. Limitations are candidly stated: test scenes were reused during method development, each model has a single training realization, no external baseline (e.g., RepNet/TransRAC or classical peak tracking) is compared, data and checkpoints were not publicly released, and the simulation uses stationary cameras, constant per-clip frequencies, and idealized inverse-dynamics control. The paper supports feasibility in this tested setting but explicitly disclaims architecture ranking, real-flight robustness, and energy-efficiency claims.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A causal pipeline of signed Farneback optical flow, a shared CNN encoder, and any of three temporal modules (TCN, LIF spiking, causal attention) achieves 92-97% exact wingbeat-count accuracy in the tested MuJoCo setting at 1.5-3.0 m.
- No architecture ranking is established for exact counts: all paired scene-bootstrap intervals for accuracy differences include zero.
- Exact totals and correctly timed events are distinct outcomes: seven far-distance spiking-model clips had correct totals despite event-timing mismatches, and most other errors concentrate within one camera frame of the evaluation boundaries.
- The study explicitly does not support claims about architecture rankings across retraining, real-flight robustness, or hardware energy efficiency.
Strengths
Tightly matched comparison design
All three families share the same spatial encoder structure and initialization hashes, loss (Eq. 6), optimizer schedule, 96-frame windows, and an identical hysteresis event counter; parameter totals differ by ~2k (116,077-118,341, Table 1). Checkpoint hashes for all 18 models were verified frozen before any new test prediction was generated (Sec. 5.1).
Event-level metrics expose what total counts hide
One-to-one matching at a 2/60 s tolerance with pooled micro-F1 (Sec. 5.2, Table 3) reveals that seven far-distance CNN-LIF clips had exact totals despite FP/FN timing errors, and Table 7 localizes nearly all remaining FN/FP within one frame of the evaluation endpoints - a concrete, reusable evaluation insight.
Careful paired uncertainty treatment
Candid and thorough scope control
Controls retained in all primary metrics
Weaknesses
Test scenes were used during method development
One training realization per condition
No external or classical baseline
Data, code, and checkpoints not released
Asymmetric provenance of the TCN baseline
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What accuracy does this paper achieve for wingbeat counting?
In MuJoCo simulation with stationary cameras, exact-count accuracy was 96.67% (TCN and Transformer) and 95.00% (spiking model) at 1.5 m, and 94.44%, 92.22%, and 95.00% respectively at 3.0 m. Paired bootstrap intervals show these differences between architectures are not statistically significant.
Is the wingbeat counting demonstrated on real flights or only simulation?
Only simulation. Scenes run in MuJoCo with idealized inverse-dynamics stabilization, stationary virtual cameras, constant wingbeat frequencies per clip, and controlled appearance changes. The authors explicitly state the results do not establish real-flight robustness.
Which temporal model performed best: spiking, convolutional, or attention?
None significantly. Exact-count differences were all statistically inconclusive. The spiking model showed a small but significant event-level F1 deficit only at 3.0 m (about -0.005 versus the others), partly because seven of its clips had correct totals but mistimed events.
Is the code or dataset from this paper publicly available?
Not in this version. The arXiv submission includes ancillary aggregate and per-clip evaluation records, but the RGB/optical-flow data and trained checkpoints were kept locally and not deposited in any public repository.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1 (schematic information flow) is assessed only from its caption; the rendered figure itself could not be analyzed from the extracted text.
- The ancillary per-clip evaluation records referenced in the availability section were not part of the extracted text, so aggregate tables could not be independently recomputed.
- Review is based solely on the supplied v1 full text; external artifacts (cited repositories, seeds, ancillary files) were not verified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.