Question's Gambit: The First Move Matters in Agentic Deep Search
Radin Hamidi Rad, Amin Bigdeli, Negar Arabzadeh, Sajad Ebrahimi, Charles L. A. Clarke, Benjamin C. M. Fung, Ebrahim Bagheri
cs.AI · cs.CL · cs.IR
Quick verdict
Clue-decomposed pre-loop retrieval lifts Pi-Serini accuracy on BrowseComp-Plus (83.1→90.5, gpt-5.5); credible, well-instrumented, but lacks component ablation.
Summary
The paper argues that a deep research agent's first retrieval move is a critical design decision for multi-clue questions. It introduces Question's Gambit, a pre-loop module that decomposes the question into clues (9.6 on average), retrieves per-clue results with BM25 after corpus-grounded query expansion (a fixed GPT-4.1), unions and deduplicates the candidate pool, and reranks it against the full question with Cohere Rerank 4 Pro, delivering a top-5 opening context before the unchanged Pi-Serini ReAct loop starts. On BrowseComp-Plus (830 questions, 100k documents), the module raises answer accuracy from 83.1% to 90.5% for GPT-5.5, 68.1% to 79.0% for GPT-5.4-mini, and 71.4% to 76.9% for DeepSeek-v4-pro, with improved calibration and gold-document recall at the preview (72.9→85.9) and behavior (56.1→75.9) stages, at roughly comparable search-call counts, for about $0.05 and 28.5 seconds per question. On a 200-question MultiHop-RAG sample, gains shrink to 0.5–1.0 points. Significance rests on unpaired two-proportion z-tests (p ≤ 0.0116, Bonferroni-safe) because baseline per-question outputs are unavailable. Main limitations: no component ablation isolating decomposition, expansion, or reranking; a system prompt that steers the agent toward the first-move tool; small repeated-run samples; and dependence on proprietary APIs.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A clue-aware first-move retrieval module substantially improves deep-research agent accuracy without modifying the agent loop or retriever (+7.4 pt for GPT-5.5 on BrowseComp-Plus, Table 1).
- Gains persist through the trajectory: for GPT-5.5, previewed gold recall rises from 72.9 to 85.9 and behavior gold recall from 56.1 to 75.9 (Table 2).
- The intervention's cost is bounded at roughly $0.05 and 28.5 s median per question, with agent-issued search calls roughly unchanged (Tables 4 and 6).
- On a conventional multi-hop control (200-question MultiHop-RAG sample), the module transfers with small positive gains of 0.5–1.0 points (Table 3).
- Residual GPT-5.5 errors are dominated by evidence selection and synthesis, not retrieval coverage (only 3.8% of 79 errors lack surfaced gold; Table 7).
Strengths
Clean isolation of the intervention
The retriever (BM25/Pyserini) and the downstream Pi-Serini loop are held fixed; only the first observation changes (Sec. 3.3, Sec. 4.3). A matched-budget check on the 50-question dev subset (Sec. 5.4) shows the loop itself is unchanged: median API cost $0.23 in both systems, search calls 12.2→10.4.
Trajectory-level evidence, not just end accuracy
Table 2 tracks gold recall through preview (72.9→85.9 for GPT-5.5) and behavior (56.1→75.9) stages; Table 7's funnel over the 79 GPT-5.5 errors shows only 3.8% fail at surfacing while 55.7% surface-but-never-preview — an honest diagnosis that the residual bottleneck is evidence selection, not retrieval.
Transparent cost and tool-use accounting
Adequate inferential statistics with honest caveats
Weaknesses
No component ablation
Prompt-steering confound
MultiHop-RAG gains are marginal and statistically untested
Thin repeated-run analysis
Proprietary, non-deterministic dependencies
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper (arXiv 2609.14412) targets deep research agents on reasoning-intensive, multi-clue questions. Its central claim: the agent's first retrieval move is a critical design decision, and a pre-loop module — Question's Gambit — that decomposes the question into clues (9.6 on average), retrieves per-clue results, unions them, and reranks against the full question, yields an opening context that lifts accuracy without touching the agent loop. On BrowseComp-Plus (830 questions, 100,195 documents), integrated into Pi-Serini with a fixed BM25 backend, answer accuracy improves from 83.1% to 90.5% for GPT-5.5, 68.1% to 79.0% for GPT-5.4-mini, and 71.4% to 76.9% for DeepSeek-v4-pro (Table 1), with calibration error falling (15.7→7.58 for GPT-5.5) and preview/behavior gold recall rising (72.9→85.9, 56.1→75.9; Table 2). A 200-question MultiHop-RAG sample serves as a control (+0.5 to +1.0 pt; Table 3).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Question's Gambit?
It is a 'first move' module for LLM-based deep research agents. Before the agent's normal iterative search loop starts, the question is split into atomic clues (~9.6 on average), each clue is expanded using corpus feedback and searched separately with BM25, the results are merged, deduplicated, and reranked against the full question, and the top 5 documents are handed to the agent as its opening context. The agent loop itself is unchanged.
How much does Question's Gambit improve accuracy on BrowseComp-Plus?
When integrated into the Pi-Serini framework with BM25, answer accuracy rises from 83.1% to 90.5% for GPT-5.5, from 68.1% to 79.0% for GPT-5.4-mini, and from 71.4% to 76.9% for DeepSeek-v4-pro — absolute gains of 7.4, 10.9, and 5.5 percentage points over 830 questions, at roughly the same number of agent-issued searches and about $0.05 / 28.5 seconds extra per question.
Does the first move help on ordinary multi-hop questions, not just BrowseComp-style puzzles?
Only modestly. On a 200-question sample of MultiHop-RAG, accuracy improved from 76.0% to 77.0% (GPT-5.4-mini) and from 81.0% to 81.5% (GPT-5.5). The authors present this as stable transfer, though the gains are small and no significance tests are reported for them.
Is the code and data available?
The implementation is published on GitHub (github.com/radinhamidi/Question-s-Gambit) and BrowseComp-Plus is a public benchmark with a public BM25-indexable corpus. However, exact replication depends on proprietary services: GPT-5.5/GPT-4.1 APIs and the Cohere Rerank 4 Pro reranker, plus a 900-second wall-clock budget per question.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures (e.g., Figure 1) were analyzed from extracted text and captions only; visual layout could not be verified.
- The GitHub URL in the abstract was truncated by extraction ('Question-'), but the full URL appears in the body; appendices beyond A-C, if any, were not extractable.
- Baseline rows imported from other papers could not be cross-checked against their sources.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.