A Tree-based RAG Framework for Evidence-Intensive QA via Adaptive Planning and Topology-Aware Evidence Gathering
Songeun Lee, Kyungjin Min, Injae Na, Suyeong Lee, Chiyoung Kim, Woohwan Jung
cs.AI · cs.IR
Quick verdict
APT-RAG adds adaptive planning & topology-aware gathering to tree RAG for evidence-intensive QA; significant gains, but high cost and F1-speed trade-offs.
Summary
The paper proposes APT-RAG, a tree-based retrieval-augmented generation framework targeting evidence-intensive QA, where answers require synthesizing evidence from dozens to hundreds of documents (43.3 gold pages on average in MoNaCo, 13.0 in QAMPARI). The authors identify two failure modes in existing structured RAG: structural rigidity (fixed upfront plans, e.g., Plan*RAG) and topology-ignorant evidence gathering (root-only retrieval, e.g., ToQ). APT-RAG builds a reasoning tree recursively via depth-first traversal; at each node it contextualizes placeholders, adaptively plans (reusing sibling QA pairs, retrieving directly, or decomposing), gathers evidence laterally/externally/vertically, and generates answers. An evidence-guided batching module clusters sibling sub-questions with overlapping evidence (Jaccard similarity), reduced to graph coloring on a complement graph with a greedy largest-first approximation under a context-window constraint. On MoNaCo and QAMPARI with Qwen3-4B/30B backbones, APT-RAG attains the best answer F1 (e.g., 50.84 vs. Plan*RAG's 47.03 on MoNaCo-30B; retrieval recall 50.79 vs. 39.28), with paired permutation tests p≤0.001 and bootstrap CIs; only RT-RAG at 4B on a 300-example MoNaCo subset slightly exceeds its F1. APT-RAG scales retrieval with evidence demand while baselines stay flat. Main limitations: it is the most expensive method on MoNaCo (33.6 LLM calls, 104.6s per question at 30B), and its efficiency modules (lateral gathering, batching) reduce F1 relative to ablated variants.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Existing structured RAG methods fail in evidence-intensive QA due to structural rigidity and topology-ignorant evidence gathering (§1, Figure 1).
- APT-RAG adaptively expands reasoning-tree depth/breadth and retrieval volume with evidence demand, while baseline retrieval stays constant (§4.3, Figures 5-6).
- APT-RAG achieves the best answer F1 and retrieval recall across MoNaCo and QAMPARI at 4B/30B scales, e.g., 50.84 F1 / 50.79 Ret. R on MoNaCo-30B vs Plan*RAG's 47.03 / 39.28 (Table 2), with all gains p≤0.001 (Appendix I).
- Evidence-guided batched generation cuts latency (41.4% from lateral gathering, 8.2% from clustering on average; larger tail reductions, Figures 7-8) via a greedy graph-coloring approximation with proven bound relationships (Appendix E).
Strengths
Clear, quantified problem diagnosis
Intro and Figure 1 ground the two claimed failure modes (structural rigidity in Plan*RAG; topology-ignorant gathering in ToQ) with worked counterexamples, and §1 quantifies the scale gap (2.0 pages in HotpotQA vs 43.3 in MoNaCo).
Principled efficiency contribution with proofs
§3.5 formalizes sub-question batching as constrained clustering; Appendix E rigorously reduces it to clique cover/graph coloring on the complement graph, proves the bound hierarchy (K̃* ≤ K̂* ≤ K̂), and Figure 10 empirically shows the token-budget constraint costs less than 1 cluster on average.
Unusually strong statistical reporting
Mechanism analysis supports the central claim
Transparent cost accounting and full operational detail
Weaknesses
Efficiency modules sacrifice quality, under-discussed
Strongest structured competitor only on a subset
No compute-matched comparison
Controlled corpus may inflate retrieval numbers
Pipeline stochasticity unreported
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
APT-RAG (arXiv 2609.04981) is a method paper for evidence-intensive QA — questions whose answers must be synthesized from dozens to hundreds of documents (MoNaCo averages 43.3 gold pages; QAMPARI 13.0, Table 1), against ~2.0 pages for HotpotQA. The authors claim two failure modes in structured RAG — *structural rigidity* (fixed upfront plans, e.g., Plan*RAG) and *topology-ignorant evidence gathering* (root-only retrieval, e.g., ToQ) — and propose a recursive depth-first reasoning-tree framework with adaptive planning (answerability checker + decomposer), three evidence-gathering modes (lateral sibling reuse, external retrieval, vertical child aggregation), and an evidence-guided batching module that reduces LLM calls. Claims of best answer F1 and retrieval recall on both benchmarks, adaptive retrieval scaling, and latency reduction are each tied to specific experiments.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is APT-RAG?
APT-RAG is a tree-based retrieval-augmented generation framework for evidence-intensive question answering. It recursively builds a reasoning tree where each node adaptively decides whether to reuse sibling answers, retrieve documents directly, or decompose into sub-questions, with evidence flowing both laterally (between sibling nodes) and vertically (from children to parents).
How well does APT-RAG perform compared to other RAG methods?
In the paper's experiments, APT-RAG reports the best answer F1 on MoNaCo and QAMPARI with Qwen3-4B/30B backbones — for example, answer F1 of 50.84 and retrieval recall of 50.79 on MoNaCo-30B versus 47.03 and 39.28 for Plan*RAG — with improvements significant at p≤0.001 under paired permutation tests. The one reported exception is RT-RAG slightly higher F1 at the 4B scale on a 300-example MoNaCo subset.
Is code available for APT-RAG?
Yes. The paper links a public repository (github.com/hyudsl/APT-RAG), and the appendices provide all prompt templates, pseudocode, corpus construction, and indexing details needed to rerun the pipeline on the public MoNaCo and QAMPARI benchmarks.
What are the main drawbacks of APT-RAG?
It is the most expensive method evaluated on MoNaCo — about 34 LLM calls and roughly 105 seconds per question with a 30B backbone — and the paper's ablations show its efficiency-oriented modules (lateral gathering and batched generation) lower answer F1 by up to about 3 points relative to running the framework without them.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure-internal data (e.g., Figures 5-8, 10-11) were extracted as fragmented axis labels and numbers; judgments about plotted values rely on captions and in-text prose rather than the curves themselves.
- The extracted text ends naturally with the Appendix F prompt templates (Figure 22); no end-of-document truncation was observed, so no truncation caveat is asserted.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.