PARSER: Read in Parallel, Reason in Depth for Long-Context LLM Agents
Kun Li, Zexuan Qiu, Tianhua Zhang, Irwin King, Helen Meng
cs.CL
Quick verdict
Scatter-gather agents with RL lead beat sequential memory on multi-hop QA to 896K tokens; strong controls, but narrow benchmarks and thin statistics.
Summary
PARSER is a scatter–gather multi-agent architecture for long-context question answering that decouples document reading from reasoning. Frozen off-the-shelf subagents, each bound to one chunk, read all chunks in parallel under the lead agent's query; a ReAct-style lead agent—trained alone with GRPO on verifiable answer rewards—iteratively broadcasts queries, gathers evidence, and refines its search. Evaluated on synthetic RULER-style packed HotpotQA (in-distribution) and 2WikiMultiHopQA (out-of-distribution) from 7K to 896K tokens (128 questions per length), PARSER with a Qwen3.5-4B backbone averages 84.6% on HotpotQA versus 78.6–78.9% for reimplemented MemAgent/ReMemR1, and reaches 86.8% with the 9B backbone, 6.3 points above DeepSeek-V4-Pro (that gap is HotpotQA-only; on 2WikiMultiHopQA the 9B lead narrows to 0.3 points). Controlled perturbation experiments show sequential baselines swing with evidence position, order, and distance while PARSER stays flat, and amortized latency at 896K drops from 876s to 78s per sample at concurrency 1 (the advantage shrinks to 1.7× at concurrency 16, and PARSER uses two GPUs versus one). Strengths include honest failure analysis and complexity caveats; main limitations are narrow synthetic QA benchmarks, 128-question subsets with 3-run averages but no variance or significance testing, lenient bidirectional Sub_EM matching, no code-release statement in the provided text, and heavy hardware (16 H100s for training).
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Decoupling reading (parallel frozen per-chunk subagents) from reasoning (an RL-trained lead agent) beats sequential memory agents on multi-hop QA up to 896K tokens: +5.7 average and +12.0 at 896K on HotpotQA with a 4B backbone (Table 1a).
- PARSER-9B surpasses DeepSeek-V4-Pro by 6.3 points on HotpotQA average (86.79 vs 80.47), a gap that shrinks to +0.3 on out-of-distribution 2WikiMultiHopQA.
- PARSER's accuracy is nearly invariant to evidence position, order, and distance perturbations that cause large swings in MemAgent and ReMemR1 (§5.1, Figure 2).
- Parallel reading cuts amortized inference latency by up to 11× at 896K tokens under concurrency 1 (876.20s → 78.22s), with the advantage shrinking to ~1.7× at concurrency 16 (Table 2).
- Subagents can remain frozen because chunk-local evidence extraction is simple: accuracy saturates from a 4B subagent (Table 3), and the trained lead agent generalizes out-of-distribution better than trained memory baselines (Table 1b).
Strengths
Controlled perturbation experiments localize the mechanism, not just the win
§5.1/Figure 2 varies evidence position, order, and distance independently: reversing logical order collapses MemAgent 66.5→23.3 while PARSER holds 96.4→97.1; growing evidence separation (0→3,200–3,400 paragraphs) drops ReMemR1 77.2→61.0 and MemAgent 50.0→44.6 while PARSER stays ~87–89. These directly implicate fixed-capacity document-ordered compression as the failure source.
Fair, thoroughly ablated comparison protocol
MemAgent and ReMemR1 are reimplemented with identical backbones and training data (§4.2, Appendix D.3); Table 7's w/o-RL rows show PARSER's prompted workflow alone already averages 74.42 (HotpotQA, 4B) vs 45.05/45.67 for sequential baselines without RL, separating architecture gains from RL gains; Tables 3–4 ablate subagent scale (2B/4B/9B) and chunk size including a no-chunking variant (73.76 vs 84.57).
Candid efficiency analysis including its own weaknesses
Transparent failure case and metric caveat
Weaknesses
Narrow synthetic evaluation
Thin statistical reporting
Abstract-level overclaim on the DeepSeek comparison
Latency comparison asymmetries
Unquantified train/inference mismatch
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
PARSER (arXiv 2609.06702, CUHK) targets long-context multi-hop QA. Its thesis: sequential memory agents (MemAgent, ReMemR1) couple document traversal to reasoning, inheriting sensitivity to evidence placement and linear latency. PARSER replaces sequential traversal with scatter–gather: T frozen subagents each own one chunk and answer the lead agent's query on it in parallel; a ReAct-style lead agent — the only trained component (GRPO, outcome reward, observation-token masking, Eq. 1) — broadcasts queries, gathers findings, and iterates (Algorithm 1; full prompts in Appendix E). Central claims: (1) +5.7 points on HotpotQA average over the strongest sequential baseline with a 4B backbone and +12.0 at 896K (Table 1a: 84.57 vs 78.91/78.58; 85.42 vs 73.44); (2) the 9B variant surpasses DeepSeek-V4-Pro by 6.3 points; (3) robustness to evidence position/order/distance perturbations; (4) up to 11× lower amortized latency at 896K.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is PARSER and how does it work?
PARSER is a multi-agent architecture for answering questions over very long documents. Many small frozen subagents each read one chunk of the document in parallel, while a single trained 'lead agent' iteratively broadcasts queries to all of them (scatter-gather), collects their evidence, and decides what to ask next until it can answer. Only the lead agent is trained, using reinforcement learning (GRPO) on verifiable answer correctness.
How much faster is PARSER than MemAgent?
At 896K-token documents with one concurrent request, PARSER's amortized latency is 78s per sample versus 876s for MemAgent (about 11×). At concurrency 16 the advantage shrinks to about 1.7× (59s vs 102s). Note PARSER uses two GPUs (one for subagents, one for the lead agent) while the baselines use one.
What data was PARSER evaluated on?
Synthetic multi-hop QA benchmarks in the RULER style: questions from HotpotQA (in-distribution) and 2WikiMultiHopQA (out-of-distribution), each padded with random distractor Wikipedia paragraphs to lengths from 7K to 896K tokens, with 128 questions per length setting.
Does PARSER beat frontier long-context models like DeepSeek?
On in-distribution HotpotQA, PARSER-9B averages 86.8% versus DeepSeek-V4-Pro's 80.5% (+6.3 points). On out-of-distribution 2WikiMultiHopQA the two are nearly tied (88.5 vs 88.2), so the frontier-model advantage is largely specific to the setting the method was trained on.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text is a sampled excerpt: §3.3 (reward definition start), §5.4 (compatibility/subagent-swap experiments), and parts of Appendices C–D fall in sampling gaps, so some content (e.g., results for the DCI baseline detailed in Appendix D.2) exists in the full paper but was not visible here.
- No explicit code-release statement appears in the provided text; only a project page URL is given, so codeAvailable=false reflects the visible text, not a verified absence in the full paper.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.