Whom Do AI Agents Work For? Role Assignment Induces Sponsorship Bias in LLM Recommenders
econ.GN · cs.AI
Quick verdict
LLM agents weaken the sponsorship penalty when the platform, not the consumer, is their principal; a well-powered, multi-model result from a single domain.
Summary
The paper asks whether an LLM shopping agent's evaluation of sponsored listings depends on which party is named as its principal in the system prompt. Grounding the question in the fiduciary concept of conflict of duty, the authors set a procedural-fairness benchmark: identical listings must receive identical evaluations regardless of the delegating principal. In Study 1, a 2x2 Monte Carlo experiment (500 zero-shot replications per cell; Gemini 3.1 Pro; matched hotel-choice tasks with randomized positions and price-matched listings) shows a consumer-delegated agent penalizes a sponsored listing by 50.2 percentage points, while platform delegation attenuates the penalty to 29.2 points. Reasoning-trace scoring by a human-validated LLM judge (ICC=.950) links the attenuation to reduced skepticism, supporting moderated mediation. Effects replicate across five further models (Claude Sonnet 5, GPT 5.6 Terra, three Gemini Flash generations), prompt-wording variants, and low vs. high reasoning effort. Study 2 (N=4,000) decomposes the label: platform attribution, not paid-placement disclosure per se, drives the divergence ('Promoted by the platform': 74.2% vs. 34.6% target choice), and stricter 'Sponsored' wording lowers overall selection but does not close the principal gap. Limitations: a single hotel-booking domain, mediation based on generated text, no human baseline, and no code/data availability statement.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
4 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- An LLM agent's evaluation of a sponsored listing depends on the named principal: the sponsorship penalty falls from 50.2 to 29.2 percentage points under platform vs. consumer delegation.
- Platform delegation weakens the skepticism that disclosure triggers in pre-choice reasoning traces, and this moderated pathway statistically accounts for part of the attenuated penalty.
- The bias is amplified specifically when the disclosure attributes the paid placement to the platform ('Promoted by the platform': 74.2% vs. 34.6% target choice).
- Stricter terminology ('Sponsored' vs. 'Promoted') lowers overall selection of paid listings but does not close the gap between platform- and consumer-delegated agents.
- Disclosure mandates designed for human consumers cannot by themselves protect consumers when evaluation is delegated to an AI agent.
Strengths
High-powered, confound-controlled experimental design
500 zero-shot replications per cell (N=2,000 in Study 1), target and competitor price-identical (P ~ U($173.45, $351.40)), position order randomized, dominated alternatives never chosen (reducing the task to a clean binary choice), and 36 stimulus fixed effects; Web Appendix B reports an explicit power analysis (16 events per parameter, worst-case MoE +/-8.8 pp).
Unusually broad robustness battery
H1/H2 replicate across six LLMs from three providers (Table 8, all Principal x Sponsorship interactions p<=.007), two prompt-wording variants that remove the action-question mismatch and the lexical overlap between 'hotel booking platform' and the 'Platform Sponsor' tag (Table 10), and LOW vs. HIGH thinking level with a non-significant three-way interaction (beta=0.84, p=.104, Table 11).
Validated measurement of the mechanism
Study 2 isolates the semantic trigger with policy relevance
Weaknesses
Normative benchmark is asserted, not demonstrated
Mediation rests on generated text scored by another LLM
External validity confined to one narrow task
Reproducibility depends on volatile proprietary APIs
Mechanism evidence is Gemini-only
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is sponsorship bias in LLM recommenders?
In this paper it means an AI shopping agent evaluates sponsored listings more favorably when told its principal is the platform rather than the consumer. With objectively identical listings, the penalty applied to a sponsored hotel fell from 50.2 percentage points under consumer delegation to 29.2 points under platform delegation, an effect replicated across six models from three providers.
Do 'Sponsored' labels still protect consumers when an AI agent shops for them?
Only partially, according to this study. Stricter wording ('Sponsored' instead of 'Promoted') sharply lowers how often the agent picks the paid listing, but when the label names the platform, platform-delegated agents still choose it about six times more often than consumer-delegated agents (21.2% vs. 3.6%), so disclosure does not close the gap created by who deployed the agent.
Which AI models were tested and how was the effect measured?
Gemini 3.1 Pro was the main subject, with replications on Claude Sonnet 5, GPT 5.6 Terra, and Gemini 3.5/3.6/3.8 Flash. Each experimental cell used 500 independent zero-shot runs of a matched hotel-choice task where sponsored and organic options had identical prices and positive reviews; effects were estimated with logistic regressions including hotel and review fixed effects, and mechanism was probed with an LLM judge validated against a human coder (ICC=0.95).
What drove the bias: the word 'sponsored' or the mention of the platform?
The platform attribution. In Study 2, adding 'by the platform' to the label roughly tenfold multiplied the odds ratio between the two agents ('Promoted by the platform': 74.2% target choice under platform delegation vs. 34.6% under consumer delegation), while the disclosure-type component ('Promoted' vs. 'Sponsored') mainly shifted overall selection for both agents.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures are not analyzable from the extracted text (captions only); cross-LLM choice shares were verified against Table W1 instead of Fig. 2.
- The judge prompt template (Web Appendix C) is summarized rather than fully reproduced in the extraction, so judge-prompt sensitivity cannot be assessed.
- Model identifiers cited (e.g., Gemini 3.1 Pro, GPT 5.6 Terra, Ornith 1.0 35B) could not be independently verified from the text alone.
- No data/code availability statement appears in the extracted text; one may exist in a portion not included.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.