SFT or RL for Tool-Calling Agents? A Controlled Study Across Data, Method, and Scale
Md Tahmid Rahman Laskar, Xue-Yong Fu, Shashi Bhushan TN
cs.CL
Quick verdict
A controlled SFT-vs-GRPO-vs-SFT→GRPO study across six Qwen3 scales and three tool-calling corpora concludes LoRA-SFT is the strongest in-distribution method and dataset choice matters as much as method; the design is unusually thorough, but near-one-point transfer margins lack significance testing and the eval metric exactly equals the RL reward, biasing the comparison toward GRPO—making the 'GRPO modestly better at transfer' claim soft.
Summary
This empirical study compares three post-training strategies—LoRA SFT, GRPO, and SFT followed by GRPO—for tool-calling agents across six Qwen3 models (0.6B–32B) and three public corpora (xLAM/APIGen, ToolACE, Glaive-FC-v2) plus a uniform mixture, evaluating exact-match accuracy on held-out test splits of all three datasets. The headline results: SFT with LoRA is the strongest in-distribution method in 15 of 18 settings and never drops below the zero-shot baseline; GRPO wins 29 of 54 cross-dataset transfer settings but its average margin over SFT is under one point; the SFT→GRPO pipeline is rarely best under a matched data budget; and training-corpus choice swings accuracy by up to 55 points, with the mixture offering the best transfer-robust default. LoRA also beats full-parameter fine-tuning at 4B. Strengths include the scale of the factorial design, contamination-aware deduplication (58.5% of Glaive removed), and reward-metric parity. Key limitations: no statistical significance tests despite sub-point margins and 1,000-example test sets; no compute-matched SFT baseline; evaluation uses the same decomposed scoring function as the GRPO reward; single model family; the LoRA-vs-full-FT comparison is limited to one scale; and Table 2 contains anomalies (e.g., SFT→GRPO transferring worse than zero-shot on some settings) without discussion.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- SFT with LoRA is the strongest in-distribution post-training method across all six Qwen3 scales (0.6B–32B), winning 15 of 18 settings and never falling below the zero-shot baseline.
- GRPO's advantage is limited to cross-dataset transfer, where it wins 29 of 54 settings but improves over SFT by less than one point on average.
- Under a matched data budget, the SFT→GRPO pipeline is rarely the strongest strategy for either in-distribution performance or transfer.
- Training-corpus choice affects in-distribution accuracy by up to 55 points, and a seeded uniform mixture is the best deployment default, combining near-specialist in-distribution scores with the strongest transfer.
- LoRA outperforms full-parameter fine-tuning (shown at 4B), consistent with LoRA better preserving pretrained agentic behavior.
Strengths
Unusually complete factorial design
6 model scales × 4 training corpora × 3 methods, each model evaluated on all three held-out test splits (Table 2, 324 trained cells plus 18 zero-shot baselines), enabling in-distribution and cross-dataset transfer claims to be separated rather than conflated as in ToolRL or Nemotron-Tool-N1.
Contamination-aware data preparation
Near-duplicates are removed on normalized queries and tool names before splitting (Section 3.1), eliminating over half of Glaive (65,256 examples, 58.5% in Table 1) — a level of deduplication rigor few training-comparison papers report.
Reward-metric parity and prompt control
Honest, quantified headline claims
Deployment-relevant practical guidance
Weaknesses
No statistical significance testing
Evaluation metric is identical to the RL reward
No compute-matched comparison
Unexplained anomalies in Table 2
Narrow model family and single-scale LoRA-vs-full comparison
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Is SFT or RL (GRPO) better for training tool-calling LLM agents?
According to this controlled study across six Qwen3 models (0.6B–32B) and three public tool-calling datasets, supervised fine-tuning with LoRA is the strongest method when training and test data come from the same distribution — it wins 15 of 18 settings and never underperforms the zero-shot model. GRPO reinforcement learning only holds a modest edge on cross-dataset transfer (winning 29 of 54 transfer settings), and even there its average advantage over SFT is under one accuracy point.
Does combining SFT and GRPO (SFT then RL) help tool-calling performance?
In this paper, the SFT→GRPO pipeline is rarely the best strategy. Under a matched data budget (each stage sees a disjoint half of the training set), it wins only 1 of 18 in-distribution settings and 6 of 54 cross-dataset transfer settings. In a few cases it even falls below the zero-shot baseline, so the authors do not recommend it as a default.
Which tool-calling dataset is best to fine-tune on: xLAM, ToolACE, or Glaive?
The paper finds dataset choice can swing in-distribution accuracy by up to 55 points. Glaive-trained models transfer poorly (below the zero-shot baseline on xLAM), while the uniform mixture of xLAM, ToolACE, and Glaive (capped at 20,000 examples per source) gives the best overall default: within 1–5 points of the best specialist on every target dataset and the strongest cross-dataset transfer, regardless of training method.
Does LoRA work as well as full fine-tuning for tool-calling models?
Yes, and better: at the 4B scale tested, LoRA (r=32, α=64) outperformed full-parameter fine-tuning on all three test sets (e.g., 88.0% vs 80.4% on xLAM), and was the only approach that never dropped below the zero-shot baseline. The authors attribute this to LoRA better preserving pretrained agentic behavior, consistent with the 'LoRA learns less and forgets less' finding — though this comparison was run at a single model scale.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Review is based on the extracted full text including main sections, Table 2, and appendices A.1–A.5; Figure 1 numbers were readable in text form, but any figures beyond it (none appear) could not be visually inspected.
- The BFCL/API-Bank external-harness results referenced in Appendix A.1 are not present in the extracted text, so it is unknown whether external-benchmark results exist and contradict or support the main matrix.
- Table 2 highlight colors (bold/blue/orange/green/red) could only be partially verified from extracted text; at least one bold assignment (32B Mix→Glaive SFT→GRPO 40.3) appears inconsistent with the described highlighting scheme, suggesting possible extraction or captioning errors.
- No per-run variance or seed information beyond the single fixed data-split seed was available in the text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.