PersuaRL: Reinforcement Learning-Driven Multi-Expert Selection for Persuasive Dialogue Generation in Insurance
Rohan Kirti, Akash Ghosh, Aryan Vats, Niladri Ghosh, Shipra Shriparn, Roshni Ramnani, Anutosh Maitra, Sriparna Saha
cs.CL
Quick verdict
Claims GRPO expert selection boosts persuasive insurance dialogue; in-domain gains credible, OOD transfer marginal, eval-judge overlap a concern.
Summary
The paper introduces PersuaRL, a reinforcement-learning framework for persuasive dialogue generation in motor insurance, plus InsureDial, a synthetic dataset of 1,931 multi-turn dialogues (26,766 utterances) built with GPT-4o and human filtering, annotated for persuasion strategy, intent, sentiment, and key terms. PersuaRL treats per-turn expert selection as an RL action: a lightweight selector trained with GRPO chooses among four fine-tuned experts (engagement, intent, keyterm, sentiment), and a generator is SFT-trained on the highest-reward selections in alternating optimization. Rewards combine strategy/intent consistency (BERT classifiers), contextual similarity, non-repetitiveness, and an LLM judge (Prometheus-2-7B). On InsureDial, PersuaRL beats single-shot and SFT variants of Llama-3.2-3B, Qwen2.5-3B, Phi-3-mini, and Mistral-24B (e.g., Llama BF1 0.742 to 0.771; human Persuasive Effectiveness 2.46 to 4.36), with 3B models surpassing 14B-70B vanilla baselines. On out-of-domain DEAL, gains are marginal and not uniformly monotonic (Llama BF1: 0.552 SFT vs 0.536 PersuaRL), and the authors' own error analysis shows insurance language leakage. Statistical reporting is thin (blanket 5% t-test, single seed, no variances), the evaluation judge overlaps the R5 reward, and no external persuasive-dialogue system is compared. Limitations — synthetic data, offline gold-history evaluation, action-space scalability — are candidly acknowledged.
Review complete
Assessment at a glance
6
risk flags identified
● high 1
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A GRPO-trained selector choosing among four fine-tuned experts per turn, with an alternating SFT generator, improves persuasive dialogue quality over single-shot and SFT baselines on InsureDial across 3B-24B backbones.
- PersuaRL on 3B models outperforms much larger (14B-70B) vanilla baselines on InsureDial automatic metrics (e.g., BF1 0.771 vs Phi-3-Medium-14B's 0.655).
- The framework generalizes out-of-domain with consistent monotonic Single-to-SFT-to-PersuaRL gains on the DEAL tourism-negotiation dataset.
- InsureDial is the first persuasive dialogue dataset for motor insurance, annotated across strategy, intent, sentiment, and key terms via a human-in-the-loop GPT-4o pipeline.
- Learned reward-driven expert selection outperforms uniform expert activation (AllExpert) and prompt-based tool routing.
Strengths
Principled alternating-optimization design
The selector's GRPO step (G=8 rollouts, KL beta=0.04, LoRA r=16, App. D.2) scores masks against a frozen generator so rewards stay stationary; the generator only ever SFTs on gold responses under the best mask. FAQ 6-7 give a credible argument that the generator cannot exploit reward artifacts, since RL touches only the discrete mask.
Unusually complete ablation suite
Expert removal (Table 3: dropping Engagement drops B2 0.375 to 0.284), reward removal (Table 11: full 0.375 vs minus-all 0.323), AllExpert with untrained vs trained generators (Tables 14-15), human evaluation of both selector-free variants (Table 17), and base-model ablations (Tables 13, 16) jointly isolate selector, generator, experts, and rewards.
New annotated in-domain resource with documented pipeline
Consistent in-domain gains across backbones and evaluation types
Candid limitations and concrete error analysis
Weaknesses
OOD robustness claim contradicted by the paper's own table
Statistical reporting too thin for the significance claim
Evaluation judge overlaps the training reward
Closed synthetic loop and missing external baselines
Mismatched criterion for reward-weight selection and unvalidated R1 surrogate
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper targets persuasive multi-turn dialogue generation for motor insurance. It contributes (i) InsureDial, a 1,931-dialogue / 26,766-utterance dataset (Table 1: 1,545/97/289 split) drafted by GPT-4o from 50 human seed dialogues, human-filtered, and four-dimension annotated (strategy, intent, sentiment, key terms) via Gemini-2.0-Flash pre-labels plus human verification; and (ii) PersuaRL, which reframes per-turn expert selection as an RL action: a lightweight GRPO-trained selector emits a binary mask over four fine-tuned experts (engagement, intent, keyterm, sentiment), whose outputs are packed with context and decoded by a generator LLM. Headline claims: PersuaRL beats single-shot and SFT baselines across four backbones (3B-24B) on InsureDial; 3B variants outperform 14B-70B vanilla models; selection transfers out-of-domain to DEAL; and learned selection beats AllExpert and prompt-based routing.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is InsureDial and how was it built?
InsureDial is a motor-insurance persuasive dialogue dataset with 1,931 multi-turn dialogues (26,766 utterances). GPT-4o drafted conversations guided by 50 human-written seed dialogues, human annotators filtered them, and Gemini-2.0-Flash pre-labels were human-verified across four dimensions: persuasion strategy, user intent, sentiment, and key domain terms. The split is 80/5/15 train/validation/test.
Does PersuaRL beat larger models like GPT-5 or 70B models?
On InsureDial, PersuaRL on small backbones beats the un-finetuned large models on overlap metrics — e.g., Llama-3.2-3B PersuaRL reaches BERT-F1 0.771 versus 0.655 for Phi-3-Medium-14B and 0.610 for Llama-3.3-70B — and PersuaRL-Mistral-24B tops the table at BF1 0.873. GPT-5 and GPT-4.1-mini have no LLM-judge scores (omitted to avoid judge bias) and score low on n-gram overlap, a known weakness of frontier chat models on this metric family.
Does the method generalize outside insurance?
Only weakly, judging by the paper's own data: on the tourism-negotiation DEAL set, gains over SFT are small and not uniformly monotonic (Llama BF1 goes from 0.552 with SFT to 0.536 with PersuaRL), and the authors' error analysis (Table 18) shows insurance phrasing like 'coverage' and 'premium' leaking into tourism responses.
Is the code and data available, and what does reproduction require?
The paper states that code and dataset are available via a link labeled 'PersuaRL'. Training used a single A100 80GB GPU for 25-28 hours per model with GRPO group size 8 and LoRA r=16, so exact reproduction is GPU-bound; the generation and expert prompts are listed in the appendix.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text contains two '[...sampling gap...]' cuts; content inside them (parts of Sections 4.2-4.3, one reference block, and appendix material around D.3.3-D.3.4 including Table 12) was skipped by sampling and was not directly assessed — judgments about those parts rely on surrounding narrative only.
- The code/dataset link appears in the text only as the word 'PersuaRL' with no resolvable URL, so artifact availability is taken from the paper's statement, not verified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.