When LLM Agents Fail to Read the Room: ReAdapt for Relational Social Reasoning
Jianzhe Lin, Jianzhe Lin, Xiaolin Li, Xiaolin Li, Yunda Liu, Fei Wang, Fei Wang, Jubin Chheda
cs.AI
Quick verdict
Claims explicit social-state updates lift LLM-agent relational accuracy (37→51%, 69→77%); credible proof-of-concept but single model, no ablation/stats.
Summary
The paper argues that LLM agent loops such as ReAct can acquire social evidence through tools yet lack an explicit mechanism for letting that evidence revise a pending decision ('evidence-to-decision coupling'). The authors build a benchmark of 500 procedurally generated social worlds (profiles, friendships, follows, reaction histories, feeds) yielding 1,000 queries over two tasks — reaction selection and warm introduction — scored by a relationship-grounded oracle whose weights are disclosed design parameters. In about 53% of queries the surface-obvious candidate differs from the oracle ('overturn' cases), isolating hypothesis revision. ReAdapt augments ReAct with a structured state (goal, belief, relationship, norm, disclosure), an explicit Adapt update after each observation, and typed policy operations (continue/switch/abandon/clarify). With Gemini-3-Flash at temperature 0 on a stratified 300-query subset (150 per task), accuracy improves from 37% to 51% on warm introduction and 69% to 77% on reaction selection; oracle regret falls from 0.260 to 0.152 and 0.095 to 0.053; overturn cases gain up to 16 points. Limitations: a single model, one run per query, no significance tests or confidence intervals, no component ablations, imperfect compute matching (ReAdapt uses more tool calls and tokens), and results are proof-of-concept on synthetic worlds with a designed oracle; no code or prompt release is stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Standard ReAct-style loops leave evidence-to-decision coupling implicit, so an agent can retrieve decisive relational evidence yet retain an initially plausible choice.
- A benchmark of 500 synthetic social worlds and 1,000 queries, with ~53% overturn cases, isolates the ability to revise a surface-level hypothesis using relational evidence.
- With Gemini-3-Flash on 150 queries per task, ReAdapt improves accuracy from 37% to 51% on warm introduction and from 69% to 77% on reaction selection versus ReAct.
- ReAdapt reduces benchmark-oracle regret from 0.260 to 0.152 (warm introduction) and from 0.095 to 0.053 (reaction selection).
- Gains appear on both overturn and straight cases, suggesting structured state supports evidence integration beyond explicit reversals.
Strengths
Well-posed diagnostic construct
The overturn subset (Section 2.3, ~53% of queries) cleanly separates surface-following from relational hypothesis revision, and Appendix A validates the harness with scripted responders reproducing the expected 100%/0% patterns — real software-level sanity checks.
Careful, honest claim scoping
The paper repeatedly disclaims that ReAct cannot reason (Section 1), marks Eq. 7 oracle weights as design parameters rather than human utility (Section 2.2), concedes Goal/Norm/Disclosure are not independently validated (Section 3.2), and frames everything as proof-of-concept (Section 4.1); Appendix C is an unusually candid limitations list.
Regret adds severity granularity
Controlled protocol contrast
Transparent resource accounting
Weaknesses
No statistical inference
Compute confound acknowledged but uncontrolled
No component ablation
Oracle and heuristic circularity/underspecification
Reproducibility gaps
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper targets a specific failure mode of tool-using LLM agents in social contexts: *evidence-to-decision coupling*. The authors argue that a standard ReAct loop (Yao et al., 2023a) makes observations available in context but imposes no explicit transition from a current social hypothesis to a revised one, so an agent can mention decisive evidence and still retain its initial choice (Sections 1, 3.1). Three contributions are claimed: (1) a relationship-reasoning benchmark of 500 procedurally generated social worlds and 1,000 queries over two tasks — reaction selection and warm introduction — with an overturn diagnostic flagging the ~53% of queries where the surface-obvious candidate differs from the relationship-grounded oracle; (2) ReAdapt, a prompted extension of ReAct maintaining a five-slot structured state z = (Goal, Belief, Relationship, Norm, Disclosure), running a typed Adapt step after every observation and emitting one of four policy operations (continue/switch/abandon/clarify) before the next action (Section 3); (3) a controlled proof-of-concept evaluation with Gemini-3-Flash on a stratified 300-query subset reporting +14 points on warm introduction (37% to 51%), +8 on reaction selection (69% to 77%), and oracle-regret reductions of 41.5% and 44.2%. The scoping is notably careful: the authors disclaim that ReAct cannot reason, mark Eq. 7's weights as design parameters rather than human utility, and concede the Goal/Norm/Disclosure dimensions are not independently validated.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is ReAdapt in one sentence?
ReAdapt is a prompting-based extension of the ReAct agent loop that, after every tool observation, explicitly updates a structured social state (goal, belief, relationship, norm, disclosure) and emits one of four policy operations — continue, switch, abandon, or clarify — before choosing its next action.
What is the 'overturn' subset in this benchmark?
It is the approximately 53% of the 1,000 benchmark queries where the surface-obvious candidate (favored by salient content or local structure) differs from the benchmark's relationship-grounded oracle answer, so success requires the agent to revise an initially plausible choice using relational evidence.
How much does ReAdapt improve over ReAct?
Using Gemini-3-Flash on 150 queries per task, warm-introduction accuracy rose from 37% to 51% and reaction-selection accuracy from 69% to 77%; benchmark-oracle regret fell from 0.260 to 0.152 and from 0.095 to 0.053, with up to a 16-point gain on overturn cases.
Are the reported improvements statistically significant?
The paper reports no significance tests, confidence intervals, or repeated runs; each of the 300 evaluated queries was executed once at temperature 0, so the gains lack formal uncertainty quantification even though they are large in absolute terms.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1 appears only as a caption; its visual trajectory example could not be analyzed from the text.
- Exact prompts and generator hyperparameters are absent from the extracted text; they may or may not exist in the original PDF, so implementation-level judgments are provisional.
- Overturn/straight subset sizes within the evaluated 300 queries are not stated anywhere in the text, limiting statistical assessment of subset-level results.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.