DualSQL: Text-to-SQL with Multi-Agent Reinforcement Learning
Shijie Chen, Yu Gan, Yeounoh Chung, Jiani Zhang, Quannan Li, Sravan Babu Bodapati, Cody J. Greer, Yu Su, et al.
cs.CL · cs.AI · cs.DB
Quick verdict
Joint multi-agent RL on one Qwen3 backbone lifts Text-to-SQL to 71.1% EX on BIRD-Dev; ablations are solid, but the margin over 32B baselines is thin.
Summary
DualSQL is a Text-to-SQL system that unifies schema linking and SQL generation as two agents sharing one Qwen3 (4B/8B) backbone and a scaffold with three SQLite tools: a SQL executor, FTS5 full-text search, and an offline LLM-built database profiler. The pipeline is trained end-to-end with GRPO on VeRL using asynchronous rollouts, guardrails against multi-agent format collapse (strict format checks, rollout cutoff, loss masking), and a new correctness metric, REX, which handles duplicates, row order, and column permutations via optimal matching and serves as the SQL reward. Training uses 3,755 curated BIRD examples: 2,715 items solvable by Qwen3-8B and 466 that Gemini-2.5-Pro never solves are removed, and 384 gold-label corrections are added. DualSQL-8B reaches 71.1% EX on BIRD-Dev (68.6 REX), above Arctic-Text2SQL-R1-32B (70.5), and 83.1% EX on Spider's test set. Same-recipe ablations attribute +2.3 EX to agentic tool use and +1.5 EX to joint multi-agent training; oracle-schema runs show +5.4 EX headroom. Main limitations: the SOTA margin is near run-to-run variance, REX's validity is not human-validated, and the effect of training-data filtering on difficulty is unquantified.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A single shared LLM backbone driving both schema-linking and SQL-generation agents enables joint multi-agent RL that beats separately optimized pipelines (+1.5 EX, +1.9 REX over the single-agent variant on BIRD-Dev).
- Agentic rollout guardrails (strict format checks, rollout cutoff, loss masking) prevent the format collapse that importance-sampling corrections cannot, enabling sustained multi-agent RL training.
- REX is a more faithful SQL-correctness signal than execution accuracy, and using it as the reward yields models better under both metrics (Table 5).
- With only 3,755 training examples, DualSQL-8B reaches 71.1% EX on BIRD-Dev, exceeding prior single-model systems including a 32B-parameter model (70.5%).
- Schema linking leaves measurable headroom (oracle linker adds +5.4 EX for the 8B model), even though only 15.3% of observed errors stem from linking.
Strengths
Controlled same-recipe ablations isolate each design choice
Table 4 and Sec 5.2.3 hold data, reward, and hyperparameters fixed across single-turn (67.3 EX), single-agent tool-use (69.6), and multi-agent (71.1) 8B settings, attributing +2.3 EX/+5.4 REX to agentic tool use and +1.5 EX/+1.9 REX to joint multi-agent training; Table 5 shows switching the reward from EX to REX improves both metrics (66.7 vs 63.7 REX).
REX targets documented EX failure modes with a concrete algorithm
It handles duplicate rows, row-order insensitivity, and column permutations via LSAP-based optimal matching with explicit extra-row/column penalties and complexity analysis (Appendix E); the reward ablation supports the claim that cleaner rewards train better models even when the benchmark is scored with coarser EX.
Honest headroom and error analysis
Guardrails address an observed, specific failure mode
Efficiency and disclosure
Weaknesses
Marginal evidence for the SOTA claim
REX validity is assumed rather than demonstrated
Training-set curation effects unquantified
Hand-tuned reward and thin toolset utilization evidence
Reproducibility gaps
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What accuracy does DualSQL achieve on the BIRD Text-to-SQL benchmark?
On the BIRD development set, DualSQL-4B scores 68.0% execution accuracy (EX) and DualSQL-8B scores 71.1% EX (64.0 and 68.6 under the stricter REX metric). Both models were trained on only 3,755 curated examples from the BIRD training set, and the 8B version outperforms a prior 32B-parameter system (70.5% EX).
What is REX and how does it differ from standard execution accuracy (EX) for evaluating SQL?
REX (Robust Execution Match) compares the result sets of two SQL queries more carefully than EX: it respects row order when the question demands it, tolerates column permutations via optimal matching, and penalizes extra rows or columns. EX converts results to unordered literal sets, missing duplicates and row-order differences, and wrongly penalizes column reordering. In the paper, models score a few points lower under REX than EX on BIRD (e.g., 68.6 vs 71.1 for the 8B model).
What model and how much data was DualSQL trained on?
DualSQL uses Qwen3 4B and 8B as the shared backbone for both a schema-linking agent and a SQL-generation agent. Training uses GRPO reinforcement learning (built on the VeRL framework) over 3,755 examples curated from BIRD's training set, taking about 4.5 days on 32 H100 GPUs for the 8B model. Three database tools are available to the agents: SQL execution, full-text search, and a precomputed database profiler.
Is DualSQL's code or model checkpoint publicly available?
The paper does not announce a public code or model release. It provides training hyperparameters, the REX algorithm in pseudocode, agent prompts, and example trajectories in appendices, and it builds on permissively licensed assets (BIRD, Spider, VeRL, Qwen3), but independent replication would require re-implementing REX and the profiler.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 2 and 3 (guardrail-ablation and training curves) are images; their quantitative content could not be verified from the extracted text.
- Table 1 values were embedded in run-on prose; transcription was straightforward but layout-dependent.
- The author list in the source metadata differs slightly from the paper text (the text lists Fatma Ozcan as an author; the metadata omits her).
- Appendix F shows agent prompts but not the offline profiler LLM's prompt, so profiler quality cannot be assessed from the text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.