Mind or Message? Auditing Theory of Mind in Multi-Agent Social Simulation
Cong Li, Cheng Chen, Cheng Chen, Thomas Fung, Alex Rossi, Yi Li, Yi Li
cs.LG
Quick verdict
Claims LLM agents in negotiation track messages not minds: fluent but projection-driven, frontier-blind. Sound audit; single-reader caveat.
Summary
The paper audits whether language-model agents in social simulation maintain genuine partner models (theory of mind) or merely track the conversational record. The authors build 40 five-issue negotiation scenarios with private point tables, so both the partner's hidden preference weights and the full Pareto frontier (computed by enumeration) are exactly known. Two model families negotiate across 160 dyads under honest vs strategic disclosure; transcripts are frozen, then 2880 counterfactual probes manipulate one factor at a time (the reader's own payoff table, partner tone, an identity label, recursion direction) on byte-identical evidence. Agents agree in 96.2% of dyads, but only 0.7% of deals are Pareto optimal, 20.5% of joint value is destroyed, and the compatible issue is missed in 76.6% of deals; on two of three economic measures they are indistinguishable from a random acceptable package. Probes show that swapping only the reader's payoff table shifts the inferred partner top priority by 15.0 points (egocentric projection), a tone rewrite moves it 5.3 points, and an identity label 0.0; agents predict the partner's stated belief about them (72.5%) better than that belief tracks the truth (51.2%). Limitations: a single reader model answers all probes, scenarios are synthetic, multiple comparisons are uncorrected, and the accuracy-to-deal-quality link is descriptive only.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- LLM negotiation agents are socially fluent (96.2% agreement, 0 protocol failures) but economically near-random on structure-sensitive measures: 0.7% Pareto-optimal deals, 20.5% joint value destroyed, and the compatible issue missed in 76.6% of deals.
- Partner-model reports are egocentric: swapping only the reader's own payoff table on byte-identical evidence shifts the inferred partner top priority by 15.0 percentage points (CI [10.9, 19.1]).
- Surface cues move inferences far less than the self-anchor: a tone rewrite shifts top-priority accuracy 5.3 points and a seniority label 0.0, though the senior label inflates the estimated reservation value by 1.10 points.
- Agents track the shared conversational record better than the hidden state: second-order answers match the partner's stated belief 72.5% of the time while that belief is itself correct only 51.2% of the time.
Strengths
Exact latent ground truth by construction
Hidden weight vectors, top priority, and the full Pareto frontier are computed by enumeration (Section 3.2), with a generator self-test over 200 fresh scenarios asserting every structural property before any agent runs (Appendix A) — scoring is mechanical, not judgmental.
Frozen-transcript counterfactual design cleanly isolates causation
Probes B-E change one factor on byte-identical evidence (Section 3.4); the analyst framing for intervention B avoids the transcript-inconsistency confound, and tone re-renders are verified via numeral-multiset checks (Appendix B).
Unusually candid exact baseline
Process transparency and release
Informative contrasts, not just main effects
Weaknesses
Single unnamed reader model answers all 2880 probes
Model identities never stated in the text
No multiple-comparison correction
Synthetic, highly regular task with no human baseline
Partner models measured post hoc
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks a mechanism-level question about LLM-based social simulation: when agents negotiate, do they maintain a partner model (a representation of the other side's hidden preferences) or merely continue a locally plausible conversation? Using 40 five-issue negotiation scenarios where the partner's weight vector and the full Pareto frontier are exact by construction, and 160 dyads across two model families, the authors freeze every transcript and apply 2880 counterfactual probes that change one factor at a time on byte-identical evidence. Central claims: (1) agents are socially fluent but economically poor (96.2% agreement; 0.7% Pareto-optimal deals; 20.5% joint value destroyed; compatible issue missed in 76.6% of deals); (2) partner-model reports are dominated by egocentric projection (a 15.0-point top-priority shift from swapping only the reader's own payoff table, CI [10.9, 19.1]); (3) surface cues matter far less (tone 5.3 points; identity label 0.0); (4) agents track the shared conversational record (72.5% second-order accuracy) better than the hidden state it should reveal (51.2% first-order accuracy). The stated aim is diagnostic: a reusable validation protocol, not a verdict on social simulation.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Do LLM agents in social simulations actually model other minds?
According to this paper, not reliably. Agents answered questions about a negotiation partner's hidden preferences at 51.2% accuracy on top priority (chance 20%), and their inferences shifted by 15.0 percentage points when only the reader's own payoff table changed, while the partner's words stayed identical — evidence of egocentric projection rather than genuine inference about the partner.
How well do the AI agents negotiate in this study?
They almost always close a deal — agreement in 96.2% of 160 two-agent negotiations with zero protocol failures — but the deals are economically poor: only 0.7% of agreements are Pareto optimal, 20.5% of achievable joint value is destroyed, and the one issue where both sides' interests perfectly align is missed in 76.6% of deals. On two of three economic measures they perform like a random mutually-acceptable package.
What is the main evidence that the agents track messages rather than minds?
Three findings: (1) changing only the reader's own payoff table moves the inferred partner priority 15.0 points while a tone rewrite moves it 5.3 and an identity label 0.0; (2) readers whose own preferences happen to match the partner's look more accurate (66.6%) without doing more inference; (3) agents predict what the partner believes about them 72.5% of the time while that belief is correct only 51.2% of the time — the shared transcript, not the hidden state, is what they represent.
What are the main limitations of this paper?
All 2880 probes were answered by a single unnamed reader model, scenarios are synthetic and structurally regular (five issues, five levels, two framings), the negotiating model families are not named in the text, no multiple-comparison correction is applied, and the link between partner-model accuracy and deal quality is descriptive because models were measured after negotiations ended.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures are available only through their captions; plotted distributions (e.g., Figure 2b projection shifts) could not be inspected visually.
- Superscript formatting was lost in extraction: 'all 55 complete packages' almost certainly means 5^5 = 3125 packages; the abstract also appears truncated after the tone-effect sentence.
- Model identities may appear in parts of the paper not included in the extraction; judged unnamed based on the provided full text.
- Review is based on the submitted v1 text; no external code or data was inspected.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.