Persuaded, Not Informed: Incentive-Misaligned Witnesses Defeat In-Context Grounding
cs.CL · cs.AI
Quick verdict
Claims LLM agents are persuaded by self-interested CRM testimony, not missing info; controls are careful and honestly caveated, but causal proof is thin.
Summary
The paper identifies a failure mode in LLM agents answering operational questions over CRM records: when a sales transcript contains an assertion by a party with an incentive toward optimism (the sales representative), models treat it as evidence and clear deals that the company's own price lists and policies deem unacceptable. On 100 B2B lead-qualification tasks from CRMArena-Pro, 35 deals genuinely fail on Budget or Timeline; in 31 of these the representative claims the opposite, and models reading only the transcript accept the claim in 29 of 31 cases (94%), consistently across seven models from four providers (87-97% misled). The contribution is diagnostic rather than architectural: a bucket analysis separating persuasion from information gaps; a same-information control (supplying records raises the contain score 58 to 67 while strict accuracy falls 41 to 18 - recall up, precision collapsed); and a compute-step control isolating deterministic code as the operative component (43% vs 85% strict on GPT-4o-mini; +14-19 pooled on frontier models; +2-5 on models that already compute correctly). A pre-specified generalization test on a second task type failed because labels were underdetermined by the inputs. Limitations include a single task type and benchmark, a small contradiction set (n=31), marginal rather than paired confidence intervals, temperature-1 single samples for two models, synthetic single-turn data, and a disclosed conflict of interest: the author's employer sells software built on the advocated architecture.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- LLM agents answering CRM lead-qualification questions treat optimistic assertions by the sales representative as evidence, clearing deals the company's records deem unacceptable (29/31 contradiction cases; 87-97% misled across seven models from four providers).
- The failure is persuasion rather than missing information: only 3 of 35 genuine failures involve no assertion, and supplying the records raises recall while collapsing precision (strict 41→18, contain 58→67).
- Deterministic code computing Budget and Timeline from model-extracted facts is the operative fix, with the margin ranging from +42 on GPT-4o-mini to +2-5 on models that already compute correctly; code never lowered either metric.
- Supplying records as text to a reasoning model does not substitute for the code path and can strictly degrade accuracy through over-flagging.
- Extract-then-compute requires the policy to be exactly specified and identifiable from the inputs; the pre-specified generalization test on invalid_config failed because labels are underdetermined by the inputs.
Strengths
Diagnostic design isolates mechanism, not just scores
Three pre-specified controls (Section III): the bucket analysis (Table I: 31 of 35 genuine failures carry a false acceptability claim), the same-information control exposing a recall-up/precision-down signature (contain 58→67 while strict falls 41→18, Table IV), and the compute-step control holding one extraction fixed while varying only who computes Budget/Timeline (43% vs 85% on GPT-4o-mini, Table IV lower block).
Cross-model replication with honest uncertainty
Table II shows 87-97% misled across seven models from four providers with Wilson CIs, and 81-90% recovery by the grounded procedure for every model; the paper explicitly reads the small-n table as direction, not rates, and releases per-case outcomes for paired analysis.
Pre-registration discipline including a reported negative
Reusable evaluation postmortems
Explicit claim-strength calibration
Weaknesses
The titular causal claim is not causally established
The natural prompt-only baseline is missing
Narrow and internally inconsistent evidence bookkeeping
Disclosed conflict of interest aligned with the conclusion
Extractor vulnerability acknowledged but unquantified
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Preliminary Review: Persuaded, Not Informed (arXiv 2609.28854)
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does 'incentive-misaligned witness' mean in this paper?
A person quoted inside the model's context whose incentives favor an inaccurate answer. In these CRM tasks it is the sales representative, whose transcripts claim budgets and timelines are fine even when the company's price list and installation policy say otherwise.
Did giving the model the CRM records fix the errors?
No. Handing the records to the reasoning model improved detection of true failures (contain score 58→67) but strict exact-match accuracy fell from 41 to 18, because the model began over-flagging qualified deals - recall rose while precision collapsed.
What fix does the paper propose?
Extract-then-compute: the LLM only converts the transcript into structured facts, and deterministic code checks budget sums and timeline tiers against the catalog and policy. On the 31 contradiction cases this caught 28, versus 2 of 31 for end-to-end reasoning by the same model.
Does the fix work on every model?
Not uniformly. The margin is +42 points on GPT-4o-mini but shrinks to +2-5 on models that already compute correctly and rarely over-flag (Claude Sonnet 5, Kimi K2.6), within confidence intervals; the reliable guarantee is that code never lowered either metric. It also requires the policy to be exactly specified and identifiable from the inputs.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The abstract in the supplied metadata is truncated mid-sentence ('2-5 points on ...'); claims were cross-checked against the complete full-text version embedded in the same message.
- Figures 1-4 are available only as extracted text captions and inline bar labels; visual layout and error-bar rendering could not be verified from images.
- Some inconsistencies (Table IV caption vs text reader attribution; Fig. 4's eight models vs five in Table IV) may partly reflect text-extraction garbling, though they are consistent across multiple mentions.
- Supplementary model identifiers, extraction prompts, and per-run result files are referenced but were not part of the extracted text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.