OneLA: Scaling Linear-Attention Decoding to Large Beams in Generative Recommendation
Xiangrui Yang, Cheng Peng, Yunfeng Zhao, Liang Zeng, Ao Hu, Jiawei Yang, Shengzhe Wang, Jingshan Lv, et al.
cs.AI · cs.DC · cs.IR
Quick verdict
OneLA replaces per-beam GDN states with one shared state plus compact transition records, claiming 1.54–2.46× GR decode speedups; sound, single-model eval.
Summary
The paper targets a serving bottleneck in generative recommendation (GR), where models produce hundreds of candidate items via large-beam decoding over short semantic-ID sequences. Recurrent linear-attention models such as Gated DeltaNet (GDN) compress history into a fixed-size state, but existing serving paths materialize a full state per beam or replay shared history repeatedly, so memory and traffic grow with beam width. OneLA keeps a single prompt-derived shared state per request and represents each beam by compact append-only GDN Transition Records (α, δ, k tuples). Because each GDN step consumes the state only through two projections, the system replays records in vector form without ever materializing a per-beam matrix, while an ancestry index tracks the evolving beam tree without copying records and a fused GPU kernel reuses the shared state across beams. On a 0.8B Qwen3.5 GR model (beam width 256, prompts 1K/5K, 3–7 output tokens), OneLA reports 1.54–2.46× end-to-end decode-forward speedups over vLLM FullState and ReplaySSM. Across 405 operator shapes, median speedups reach 17.5×–46.3× depending on baseline, persistent-state capacity drops by up to 42.5× (median), and maximum output error is 4.88×10⁻⁴ versus FP32 full state. Limitations: a single model, short decode horizons (replay cost grows with depth), no released code, and unspecified hardware.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- All beams in a large-beam GR request share one prompt-derived recurrent state and diverge only through a short suffix of GDN transitions, so per-beam full-state materialization is redundant (§2.4).
- A GDN transition is fully captured by the compact record (α_t, δ_t, k_t), and projection replay over ancestral records reproduces GDN exactly without ever materializing a d_v×d_k per-beam matrix (§3.2, Eq. 2–4).
- OneLA achieves 1.54–2.46× end-to-end decode-forward speedups over vLLM FullState and ReplaySSM on a 0.8B Qwen3.5 GR workload with beam width 256 (§4.2).
- Across 405 operator shapes, OneLA attains median latency speedups of 17.5×–46.3× over five baselines, cuts median persistent recurrent-state capacity by up to 42.5×, and keeps maximum output error at 4.88×10⁻⁴ (§4.3).
Strengths
Sharp problem identification with clear workload characterization
§1–2 formalize beam width as a new scaling dimension (Θ(w·d_v·d_k) per-beam state) and exploit the GR-specific structure L_prompt≫T and W≫T, cleanly motivating one prompt-derived state shared across all beams instead of per-beam materialization.
Mathematically grounded, exact representation
§3.2 derives the GTR (α_t, δ_t, k_t) from the GDN recurrence (Eq. 2) and shows projection replay (Eq. 3–4) reproduces GDN exactly via two d_v-dimensioned vector accumulators; storing δ_t rather than (β_t, v_t) is explicitly justified, and accuracy is quantified (max error 4.88×10⁻⁴ vs FP32 FullState across all 405 operator shapes).
Clean decoupling of logical and physical state
Broad, calibrated evaluation
Mechanistic breakdown connecting representation to performance
Weaknesses
Single-model, short-horizon end-to-end evidence
Replay depth is unanalyzed beyond tiny T
Reproducibility gaps
Uneven baseline coverage
Ancestry/index overhead not isolated
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
OneLA (arXiv 2609.12399; Kuaishou/HKU/PKU) is a systems paper targeting decode-time serving efficiency for generative recommendation (GR) built on Gated DeltaNet (GDN) linear attention. The claimed problem: GR's large-beam decoding — hundreds of beams expanding short semantic-ID sequences — makes per-beam recurrent-state materialization a new scaling axis, costing Θ(w·d_v·d_k) storage plus per-step read/write traffic, and vLLM's two GDN paths handle it poorly: FullState keeps a full state per beam, while ReplaySSM checkpoints only at prompt-block boundaries and treats each beam as an independent request (§2.3). Central claims: (1) all beams share a prompt-derived state and differ only in a short transition suffix; (2) compact GTRs plus projection replay reproduce GDN exactly without per-beam matrices; (3) 1.54–2.46× end-to-end decode-forward speedups; (4) operator-level median speedups of 17.5×–46.3× across five baselines with up to 42.5× median state-capacity reduction. On inspection, the math is sound: right-multiplying the recurrence (Eq. 2) by q_t/k_t and expanding over the ancestry yields exactly the vector recurrences (Eq. 3), and storing δ_t — computed at write time — genuinely makes records parent-independent, as §3.2 argues.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What problem does OneLA solve?
In generative recommendation, models generate items as short semantic-ID sequences using large beam search (hundreds of beams). Linear-attention models like Gated DeltaNet keep a fixed-size recurrent state, and existing serving systems either store a full state copy for every beam or replay shared history repeatedly. OneLA stores one shared prompt-derived state per request plus tiny per-branch transition records, cutting GPU memory and decode time as beam width grows.
How much faster is OneLA?
The paper reports 1.54–2.46× faster end-to-end decode-forward on a 0.8B Qwen3.5-based generative recommendation model with beam width 256, 1K/5K-token prompts, and 3–7 output tokens versus vLLM's FullState and ReplaySSM paths. At the attention-operator level across 405 workload shapes, median speedups range from 17.5× (versus replay-based baselines) to about 46× (versus full-state baselines), with up to 42.5× lower median persistent state memory.
Is OneLA's computation exact or approximate?
The design is mathematically exact: projection replay reproduces the same Gated DeltaNet recurrence (Equations 2–4) without approximations. The paper reports a maximum absolute output error of 4.88×10⁻⁴ versus an FP32 full-state reference across all 405 operator shapes, attributable to floating-point operation ordering rather than any algorithmic approximation.
Is OneLA's code available?
The paper describes a prototype evaluated against vLLM, SGLang, FlashInfer, and TensorRT-LLM, but it does not state any public code, artifact, or dataset release, so independent verification currently requires reimplementing the system from the equations in Section 3.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–8 render as scattered axis-label text in the PDF extraction; all quantitative values cited in this review come from body-text numbers (§4.2–4.4), not from reading plots.
- The supplied metadata lists 8 authors, but the full text byline contains 14 (adding Xiao Liang, Chen Yang, and corresponding authors Jiaqiang Liu and Yiming Qiu); this review follows the full byline.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.