F$^{2}$DR: A Fine-Grained Full-Pipeline Reward Framework for DeepSearch Workflows
Bojian Xiong (Tianjin University), Wentao Ding (Baidu Inc.), Yujing Lu (Baidu Inc.), Shaowei Zhang (Tianjin University), Ling Shi (Tianjin University), Jing Liao (Baidu Inc.), Yan Wang (Baidu Inc.), Yueyang Zhang (Baidu Inc.), et al.
cs.CL
Quick verdict
Promising three-dimension process-level reward framework + 1,070-pair DeepSearch RM benchmark; strong gains, but circularity and verbosity confounds remain.
Summary
The paper proposes F2DR, a process-level reward framework that evaluates DeepSearch workflows — iterative plan-search-answer pipelines — along three dimensions: Content (checklist coverage and factual grounding of retrieved evidence), Trajectory (six binary sub-dimensions covering intent understanding, search atomicity, planning consistency, gap resolution, comprehensiveness, and iterative innovation), and Answer (subjective quality tiers plus checklist coverage), combined as 0.75·Content + Trajectory + 1.25·Answer. Built on an industrial DAG-based DeepSearch pipeline using DeepSeek-V3 and Qwen-Next-80B-A3B, the authors construct DeepSearch RM-Bench: 1,070 bilingual preference pairs from enterprise search queries, filtered by multi-model consensus scoring with majority voting, score-gap bounds, and human verification (Fleiss' Kappa above 0.9 across all ten sub-dimensions; an independent five-expert GSB review retained roughly 97% of pairs). Across 32 evaluated systems, F2DR lifts preference accuracy by about 10 points on average over unconstrained self-evaluation; DeepSeek-R1 reaches 85.79% versus 70.00%, while open-source reward models span 48.9–63.2%. Ablations (Table 4) show each dimension contributes independently. Main limitations: the benchmark is filtered using F2DR itself (partial circularity; DeepSeek-R1 participated in pair labeling yet tops the benchmark), chosen trajectories are systematically longer (Table 2), no significance tests or confidence intervals are reported, and data and code are not yet released.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Existing reward models and RM benchmarks, designed for static single-turn tasks, fail to evaluate full-pipeline DeepSearch workflows (§1, §2).
- F2DR's structured three-dimension evaluation raises preference accuracy by roughly 10 percentage points on average versus unconstrained self-evaluation across 16 frontier LLMs (Table 3, §4.1).
- DeepSearch RM-Bench — 1,070 bilingual preference pairs with process-level labels and human consistency above Fleiss' Kappa 0.9 — strongly discriminates among open-source reward models, which score only 48.9–63.2% (§3.3, Table 3).
- Each F2DR dimension contributes independently: removing Content, Trajectory, or Answer systematically degrades performance, with Thinking models most sensitive to Trajectory removal (Table 4).
Strengths
Genuine gap addressed with a first-of-kind benchmark
Existing RM benchmarks (RewardBench, RM-Bench, RMB) target static single-turn preferences; DeepSearch RM-Bench is the first preference benchmark over full plan-retrieve-answer trajectories, with bilingual (EN/ZH) coverage, six task types and ten vertical domains (Fig 2, §3.3).
Concrete, fully released rubrics with high human agreement
All three dimension prompts are given verbatim in Appendix B (Tables 7–9), including the 0–1.5 checklist scoring scale and six binary trajectory criteria; Fleiss' Kappa across all ten sub-dimensions is 0.9114–0.9662 (Table 1), indicating annotators can apply the rubric consistently.
Broad, informative model coverage
Explicit awareness of the self-confirmation risk
Multi-layered validation of dimension contributions
Weaknesses
Partial circularity between benchmark construction and headline evaluation
Length/effort confound in preference pairs
No inferential statistics; in-sample hyperparameter selection
Low reproducibility
Internal inconsistencies and garbled presentation
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
F2DR & DeepSearch RM-Bench — Structured Preliminary Review
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is F2DR?
F2DR is a fine-grained reward/evaluation framework for DeepSearch — iterative plan-search-answer LLM workflows. It scores a workflow along three dimensions: Content (checklist coverage and factual grounding of retrieved evidence), Trajectory (six binary planning-quality sub-dimensions such as intent understanding and search atomicity), and Answer (subjective quality tiers plus checklist coverage), combined with weights 0.75/1.00/1.25. In the paper it raises preference-judgment accuracy by about 10 points on average versus plain LLM self-evaluation.
What is DeepSearch RM-Bench?
A benchmark of 1,070 English–Chinese preference pairs (one pair per query) built from real enterprise search queries. Trajectories were generated with a DAG-based DeepSearch pipeline using DeepSeek-V3 and Qwen-Next-80B-A3B, then scored, filtered by multi-model consensus, and human-verified (Fleiss' Kappa above 0.9). It is intended to test reward models on full-pipeline DeepSearch quality; the authors state the dataset will be publicly released.
What are the main results of the paper?
Under F2DR, DeepSeek-R1 achieves the best preference accuracy at 85.79% (versus 70.00% for plain self-evaluation), followed by Qwen3-235B-A22B-Thinking (74.95%) and Qwen-Max (72.61%); GPT-5 and Claude 4.5 Sonnet reach 69.25% and 68.87%. Open-source reward models do worse: Skywork variants roughly 50–61%, Internlm2 48.87–57.47%, ArmoRM 49.43%, RM-R1 60.74–62.99%, and DeepSeek-GRM-27B 63.18%.
Is the DeepSearch RM-Bench data or code available?
Not at review time. The paper promises to release the DeepSearch RM-Bench dataset 'soon'; the queries come from a proprietary enterprise search platform and no code is released, although all F2DR evaluation prompts are reproduced in the paper's appendices, so the rubrics themselves are reusable.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–4 are extracted as garbled text; per-dimension Pearson values (Fig 3) and per-task/domain breakdowns (Fig 4) could not be independently verified from the extraction.
- GSB per-annotator totals (~1,104 judgments) are inferred from Table 15 counts; the text never states the candidate-pool denominator explicitly.
- Some figure-internal labels are corrupted in extraction ('F2DS', 'Generation Pipline'); core quantitative claims were cross-checked against prose where possible.
- Dataset release is promised but not verifiable at review time.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.