Better Together: Complementary Query Rewriting Under a Strong RAG Baseline
Sara Shanian, Xiaoqin Yi, Pavlo Ruban, Kurt MacDonald
cs.CL · cs.IR
Quick verdict
Claims rewriting helps a strong RAG baseline only via combination; careful and honest, but headline ensemble gains are single-seed post-hoc analyses.
Summary
The paper tests whether LLM query rewriting still improves RAG retrieval when the base pipeline is already strong (bge-base-en-v1.5 dense retrieval, cross-encoder reranking, MMR). It compares a no-LLM baseline (S1), three multi-query rewriting strategies (S2 neighbor, S3 synonym, S4 comparative), and two pseudo-document generators (HyDE, Query2Doc) on HotpotQA (first 5,000 train queries), AmbigNQ (full 2,002-query dev set), and EnterpriseRAG-Bench (~512K documents, 470 answerable queries), over three seeds with paired-bootstrap significance tests. Solo gains are modest (0.2–3.7pp HIT@10 over S1). The core finding is complementarity: post-hoc score-based unions beat the best solo method by +8.3 to +9.6pp HIT@10 on enterprise data (four-way 51.70, five-way 52.98 vs S1 39.22), while budget-matched controls (TopK-100, multi-pass HNSW) capture only ~40% of the gain. On AmbigNQ, fusion hurts (−2.4pp below best solo, p<0.001) because methods retrieve near-identical lists (Jaccard@10 0.55–0.99). A confidence-gated router (τ=0.65 on S1's top-1 score) captures about half the enterprise full-merge gain (+4.33 HIT@10) at 39% query expansion and improves answer F1 by +1.92 (p<0.01) in simulation. Key limitations: headline ensemble numbers are single-seed post-hoc analyses, web-QA memorization cannot be ruled out, and the enterprise result rests on one synthetic 470-query benchmark.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Under one fixed strong RAG pipeline (BGE dense retrieval, cross-encoder reranking, MMR), no single rewriting strategy dominates and solo gains over the no-LLM baseline are modest (0.2–3.7pp HIT@10, Table 3).
- Post-hoc unions of methods yield outsized complementary gains on enterprise data (four-way 51.70, five-way 52.98 vs S1 39.22 HIT@10), and budget-matched controls capture only ~40% of the gain, so complementarity rather than retrieval depth is the primary driver (Tables 5, 11).
- On AmbigNQ the same fusion hurts (−2.4pp below the best solo method, p<0.001) because methods retrieve near-identical document lists (Jaccard@10 0.55–0.99), so merging cannot create diversity it was not given (Section 6).
- A confidence-gated router on S1's top-1 reranker score (τ=0.65) captures about half the enterprise full-merge gain (+4.33 HIT@10) while rewriting under 40% of queries, self-disables on AmbigNQ, and improves downstream answer F1 by +1.92 (p<0.01) in simulation (Tables 6, 14).
Strengths
Strong-baseline design directly targets the field's weakest practice
Sections 1, 4 fix reranking (bge-reranker-base) and MMR (λ=0.85) into every configuration, explicitly because prior rewriting studies benchmark on bare bi-encoders where any rewrite looks good; Section 5.1 then shows solo gains shrink to 0.2–3.7pp HIT@10.
Budget-matched controls pre-empt the compute confound
Table 11: S1+TopK-100 (+5.11) and S1+multi-pass HNSW (+4.97) recover only 40–41% of the four-way ensemble's +12.48pp over S1 on Enterprise, isolating complementarity from retrieval depth.
Honest negative result with a mechanism
Statistical discipline throughout
Candid limitations section
Weaknesses
Headline numbers rest on a single seed
Web-QA legs of the argument are confounded and saturated
Router threshold has a circularity risk
Nonstandard HotpotQA subset and single pipeline point
Ensemble conclusions tied to one merge function
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks whether LLM query rewriting still helps RAG retrieval once the underlying pipeline is already strong. Six methods run under one fixed configuration (bge-base-en-v1.5, bge-reranker-base, MMR λ=0.85, GPT-4.1 generation): a no-LLM parent baseline (S1), multi-query rewriting (S2 neighbor, S3 synonym, S4 comparative), and pseudo-document generation (HyDE, Query2Doc), evaluated on HotpotQA (first 5,000 train queries), AmbigNQ (full 2,002-query dev set), and EnterpriseRAG-Bench (511,958 documents, 470 answerable queries) over three seeds. Four claims are made: (i) solo rewriting is only competitive (0.2–3.7pp HIT@10 gains, Table 3); (ii) unions deliver outsized complementary gains on enterprise data (+12.5/+13.8pp over baseline, Tables 4–5) that budget-matched controls capture only ~40% of (Table 11); (iii) merging hurts on AmbigNQ (−2.4pp vs best solo, p<0.001) due to high list overlap (Section 6); (iv) a confidence-gated router captures about half the enterprise merge gain at <40% rewriting cost, with +1.92 answer F1 (p<0.01, Table 14).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Does query rewriting improve RAG when retrieval is already strong?
According to this paper, alone it barely helps: under a fixed pipeline with dense retrieval, cross-encoder reranking and MMR, rewriting strategies gained only 0.2–3.7 HIT@10 points over the no-rewriting baseline. The value appears when combining several methods: a four-method union added +12.5 points on enterprise-style data (51.70 vs 39.22 HIT@10). But merging hurt on AmbigNQ (−2.4 points), where all methods retrieved nearly identical document lists.
What is the confidence-gated router proposed in the paper?
It is a cost-saving heuristic: run expensive LLM query rewriting (Query2Doc) only when the baseline's own top-1 reranker score falls below a threshold (τ=0.65); otherwise return the baseline results. In simulation it captured +4.33 HIT@10 (about half the full-merge gain) on enterprise data while rewriting only 39% of queries, and it nearly abstained on AmbigNQ where merging hurts. The authors stress it was evaluated in simulation only, not deployed.
What datasets and models were used in this study?
Three datasets: HotpotQA (first 5,000 training questions), AmbigNQ (full 2,002-question dev set), and EnterpriseRAG-Bench (~512K synthetic enterprise documents, 470 answerable questions). The fixed pipeline used bge-base-en-v1.5 embeddings, bge-reranker-base, MMR diversification, and GPT-4.1 for all query rewriting and document generation, evaluated over three random seeds.
Is the code and data for this paper available?
Yes, per Appendix A the code and experiment configurations are at github.com/ServiceNow/enhanced-rag-pipeline, and EnterpriseRAG-Bench is released on Hugging Face (onyx-dot-app/EnterpriseRAG-Bench) under an MIT license. Note that reproducing non-baseline methods requires GPT-4.1 API calls, and the enterprise index involves ~512K documents.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.