FIVE-VLA: Fast and EffectIVE Autonomous Driving with Recurrent Action Memory
Kemal Oksuz, Alexandru Buburuzan, Yuhan Yao, Puneet K. Dokania
cs.CV · cs.RO
Quick verdict
641M-param driving VLA (98 vision tokens + recurrent action memory) claims Bench2Drive closed-loop SOTA; ablations thorough, attribution and statistics thin.
Summary
The paper introduces FIVE-VLA, a 641M-parameter vision-language-action model for autonomous driving built on FastVLM (FastViTHD encoder + Qwen2-358M LLM), which converts 448x896 images into only 98 vision tokens (~5x fewer than SimLingo) and predicts trajectories in a single forward pass by appending the fixed 'Waypoints:' prefix to the input instead of autoregressively generating it. Its main novelty, Recurrent Action Memory (RAM), is a 17M-parameter pair of transformer blocks that conditions current action queries on the previous step's output action tokens, added via a second fine-tuning stage with zero-initialised projections, T=4 streams, and stop-gradients; at inference, 'RAM accumulation' swaps in a secondary memory-bearing instance every T/2 steps. On Bench2Drive closed-loop it reaches DS 90.95 / SR 77.27% vs SimLingo's 85.07/67.27 (3 seeds); on Fail2Drive generalisation HM is 71.9 vs 62.2; on the NVIDIA Physical AI AV dataset (non-reactive open-loop) NC-V drops to 0.78% vs 0.87% single-view; throughput is 3.89 fps on T4 (~8x SimLingo) and ~30 fps on A100. Ablations (Tabs. 5-8, A.13-A.15) are unusually thorough. Main limitations: roughly half the SR gain comes from the backbone swap rather than RAM; statistical reporting (seeds, tests) is limited to Bench2Drive; the open-loop safety protocol is a custom NAVSIM adaptation; no code release is stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- FIVE-VLA (641M parameters) sets a new closed-loop SOTA on Bench2Drive (SR 77.27% vs SimLingo 67.27%, DS 90.95 vs 85.07) and generalises better on Fail2Drive (HM 71.9 vs 62.2 on the OOD split).
- RAM, a 17M-parameter two-block module conditioning on previous action tokens, selectively improves temporal manoeuvres (give-way +26.67, overtaking +7.40, emergency brake +6.11 points) and driving smoothness (+10% relative) while preserving the pretrained VLM.
- Bypassing autoregressive text generation plus the FastVLM encoder yields ~8x throughput over SimLingo on a T4 (3.89 fps) and ~30x over ORION on an A100.
- On the NVIDIA Physical AI AV dataset, FIVE-VLA achieves lower collision-violation rates than SimLingo (NC-V 0.78% vs 0.87% single-view, 0.75% vs 0.82% four-view) with ~5x fewer vision tokens.
- Latent action-token recurrence outperforms explicit waypoint history (which degrades closed-loop driving due to the copycat shortcut) and vision-domain memory modules under matched training.
Strengths
Consistent closed-loop SOTA across two benchmarks
Bench2Drive: DS 90.95 / SR 77.27+-1.6 vs SimLingo 85.07/67.27 over 3 seeds (Tab. 1, A.16); Fail2Drive generalisation split HM 71.9 vs 62.2 with a smaller ID-to-OOD drop (-9.9 vs -13.2 DS, Tab. A.17) - the gains replicate rather than being benchmark-specific.
Unusually thorough ablation and control experiments
A bypass-RAM control (Tab. 8: 88.23 DS vs 88.49 no-RAM) confirms gains come from memory; training-length controls rule out longer-training confounds (Tab. A.15); explicit waypoint-history baselines cleanly disentangle the copycat problem (Tab. 5: open-loop ADE improves 0.358 vs 0.402 yet closed-loop SR drops to 64.39-70.00); robustness stress tests include wrong-action injection (Tab. A.11).
Genuine efficiency engineering with transparent accounting
Scales to real-world data and multi-view without losing the token advantage
Weaknesses
Headline gain attribution conflates backbone swap with the novel contribution
Thin statistical reporting outside Bench2Drive
Open-loop safety claims rest entirely on a custom protocol
Reimplemented temporal baselines may be under-resourced
Edge-deployment realism of the flagship speedup
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is FIVE-VLA?
FIVE-VLA is a 641M-parameter vision-language-action model for autonomous driving from Bosch/Five AI (arXiv 2609.18623). It uses the FastVLM encoder to compress a 448x896 camera image into only 98 vision tokens and predicts driving trajectories in a single forward pass without generating reasoning text, making it much faster than prior driving VLAs (~30 fps on an A100, ~4 fps on a T4 GPU).
What is Recurrent Action Memory (RAM)?
RAM is a small (17M-parameter) module of two transformer blocks that feeds the model's own action tokens from the previous time step back into the current trajectory prediction, giving the driving policy short-term temporal context useful for braking, overtaking, and giving way. At inference, a 'RAM accumulation' scheme swaps in a secondary instance so memory can build beyond the 4-step training window, improving driving score by ~2 points over disabling memory.
How much better is FIVE-VLA than SimLingo?
On the Bench2Drive closed-loop benchmark, FIVE-VLA completes 77.27% of routes without traffic-rule infractions versus SimLingo's 67.27% (about 10 points), with a driving score of 90.95 vs 85.07, while running roughly 8x faster on a T4 GPU and using ~5x fewer vision tokens. On the Fail2Drive generalisation benchmark the harmonic-mean gap widens to 9.7 points. Note that per the paper's own ablation, about half of the closed-loop gain comes from swapping in the FastVLM encoder rather than from RAM itself.
Is FIVE-VLA code or a trained model publicly available?
The arXiv paper does not state any code or weight release. It builds on public components - FastVLM, Qwen2, the SimLingo CARLA training set, the Bench2Drive/Fail2Drive benchmarks, and the NVIDIA Physical AI AV dataset - but reproducing the results would require re-implementing the two-stage training recipe and the customised open-loop (NAVSIM-style) evaluation protocol described in the appendix.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- All figures (Fig. 1-4, A.5-A.10) are images; only captions survived extraction, so qualitative claims (e.g., Fig. 4 collision timings, Fig. A.5 OOD behaviour) could not be visually verified.
- Table underline/bold markers were lost in text extraction; improvement annotations in Tabs. 1/A.16 were cross-checked arithmetically against displayed values (e.g., +10.37 overtaking = 83.70 vs PGS 73.33, the best non-LiDAR baseline) and found internally consistent.
- The review reflects a September 2026 arXiv preprint (2609.18623v1) that has not yet undergone peer review.
- Some fine-grained layout content (Fig. 2 architecture diagram labels, multi-column table alignment) was reconstructed from prose and may omit minor details.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.