MARBO: Relational Belief Grounding for LLM Agents in Social Deduction Games
Yechan Hwang, Sangjun Bae, Jeongmo Kim, Sangwoo Bang, Seungyul Han
cs.AI · cs.LG
Quick verdict
MARBO grounds KTO feedback in inferred player-role beliefs for Werewolf/Among Us agents; gains look strong but hinge on LLM-judged beliefs and hand-set rules.
Summary
The paper introduces MARBO, a preference-optimization framework that trains LLM agents for social deduction games (Werewolf, Among Us) to condition decisions and speech on explicit relational beliefs: per-player predictions of hidden roles or team affiliations. Built on KTO, MARBO adds two signals: a belief-prediction loss, and belief-gated preference labels. Task actions receive positive feedback only when the action is strategically favorable under hand-specified utility tables and the agent's belief about the target is correct; speech is labeled desirable only when an LLM-judge validity check passes and simulated listener beliefs shift in the speaker's favor. On Qwen2.5-7B/14B (Werewolf) and Gemma-E2B/E4B (Among Us), MARBO reports large win-rate gains over Base, ReCon, and MaKTO (e.g., 65.6% vs 41.6% average for 7B Werewolf; 49.0% vs 36.4% for E2B Among Us), higher belief accuracy (57.2% role prediction), and ablation drops to 34-55% without individual components. Limitations: feedback rules are heavily hand-engineered; belief shifts and shaping success are measured by LLM judges sharing machinery with training (Goodhart risk); evaluation lacks significance tests; training depends on API models; authors also note turn-only communication, hallucination, and offline-training constraints.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Gating preference feedback on correct relational beliefs (task actions) and favorable listener belief shifts (speech) yields consistent win-rate gains for compact LLM agents in social deduction games.
- MARBO outperforms prompting-based (ReCon) and preference-optimization (MaKTO) baselines across Werewolf and Among Us, five opponents per game, and multiple backbone sizes (e.g., 65.6% vs 41.6% average on Qwen2.5-7B Werewolf, Table 2).
- MARBO improves the underlying beliefs themselves: 57.2% role-prediction accuracy, 66.0% alignment, and 56.4 Wolf F1 versus baselines at or below 43.2/44.8/45.2 (Table 4).
- MARBO-7B outperforms MaKTO-14B, indicating the gains stem from belief-grounded supervision rather than model capacity.
Strengths
Belief-gated preference design is a clean, well-motivated extension of KTO
Task feedback requires both game-rule favorability and correct belief about the target (Eq. 8); speech feedback uses aggregated listener belief shifts (Eqs. 6-7). Table 7 ablations support each component: 65.6% full MARBO vs 55.0% (no belief construction), 39.0% (no belief-action), 34.0% (no belief-shaping) on Qwen2.5-7B Werewolf.
Unusually thorough evaluation apparatus for an agent-training paper
Two games, five fixed opponents plus head-to-head (Tables 2-3), role-wise breakdowns (Appendix F), belief-level metrics (Table 4), and an oracle/random-belief bracketing study (Table 5: random 30.1%, MARBO 70.2%, oracle 78.5% win rate vs GPT-4o-mini) that contextualizes the 57.2% belief accuracy.
Evidence that gains are not merely capacity
Artifact transparency
Weaknesses
Evaluation circularity between training rewards and metrics
The 'grounding' is substantially hand-engineered
Weak statistical reporting
Belief supervision may leak beyond what is inferable
Baseline set is thin and partly inverted
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
MARBO (arXiv 2609.06563v2, UNIST) is a method paper on training compact LLM agents for social deduction games. It contributes (1) an explicit relational-belief formulation (Sec 4.1, Eqs. 3-4), (2) belief-gated preference optimization built on KTO (Sec 4.2, Eqs. 5-9), and (3) empirical wins on Werewolf and Among Us. Headline numbers (Table 2): Qwen2.5-7B average win rate 65.6% vs MaKTO 41.6% (+24pp) and Base 33.2% across five opponents; 14B 72.5% vs 64.0%; Among Us Gemma-E2B 49.0% vs 36.4% and E4B 59.0% vs 40.0%. Belief-quality claims (Table 4): 57.2% role prediction, 66.0% alignment, 56.4 Wolf F1 vs baselines at or below 43.2/44.8/45.2. Table 5 usefully brackets these: random beliefs yield 30.1% win rate, MARBO 70.2%, oracle beliefs 78.5% (vs GPT-4o-mini), showing belief accuracy explains much but not all of the gain.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is MARBO in one sentence?
MARBO is a training framework that fine-tunes LLM game agents (via KTO preference optimization) to base actions and speech on explicit per-player beliefs about hidden roles, and only rewards behaviors that are both strategically favorable and supported by correct beliefs.
Does MARBO need a large, frontier LLM?
No - it targets compact open models (Qwen2.5-7B/14B for Werewolf, Gemma E2B/E4B for Among Us) and reports that its 7B agent beats the MaKTO-trained 14B baseline on average win rate (65.6% vs 64.0% in Werewolf).
Is the code and data available?
Code is released on GitHub (github.com/PleaseTakemeAway/MARBO). Training data derives from 300 random games per game plus MaKTO's public expert demonstrations, but reproduction also needs paid API models (GPT-4o family, Claude Sonnet 4.5) and 4-8 A100 GPUs.
Does MARBO generalize beyond Werewolf and Among Us?
Only these two games are evaluated. The feedback design relies on per-game, hand-written utility and belief-transition tables (Appendix B), and the authors list limits to turn-based communication and offline training, so transfer to other settings is untested.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure-only data (Figures 3-5, F.1-F.3) is available only through the prose descriptions embedded in the text; exact per-cell values of role-specific win rates were not extractable from the PDF text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.