Toward Robust Personalized Alignment for LLMs: Mitigating Persona Drift in Multi-Turn Dialogue
Youyuan Zhang, Siyuan Li, Fangming Liu, Jing Li
cs.AI
Quick verdict
CORE routes persona updates via COMMIT/DEFER/IGNORE to cut drift; broad baselines and ablations support it, but headline gains rest on a GPT-5.1 judge.
Summary
The paper argues that persona drift in long-horizon personalized dialogue partly stems from writing under-resolved turn-local observations into persistent user state. It proposes CORE, which maintains a slot-factorized persona belief with an UNK value, characterizes turn-local evidence by relevance, ambiguity, and conflict (entropy and Jensen-Shannon divergence), routes each slot-turn through COMMIT/DEFER/IGNORE, and applies gated belief revision only when authorized; a RESPOND/CLARIFY head handles unresolved evidence. Training combines supervised warm-up with PPO rewarding action correctness, state fidelity, response quality, and clarification. The authors also introduce PERSIST, a synthetic post-anchor stress benchmark with anchoring, fluctuation, and confrontation phases covering six social-influence strategies. On ALOE and PersonaChat with a Llama-3.2-3B-Instruct backbone, CORE reaches 82.94 and 76.54 AL, above the strongest baseline RLPA (75.84, 65.05); on PERSIST-Eval it improves ARUS (80.8 vs 71.9) and BCD (76.4 vs 66.1) and lowers drift (10.1 vs 12.6) versus RLPA. Ablations attribute gains to executable belief revision (removing it drops AL from 82.9 to 78.8), and Qwen2.5-3B transfer partially corroborates generality. Human evaluation with eight raters favors CORE on all five dimensions. Main limitations: headline metrics AL/ARUS are scored by GPT-5.1 while PERSIST itself is LLM-generated, raising circularity concerns; significance tests are not visible in the provided text; main results cover only 3B-scale backbones.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Premature writes of turn-local evidence into persistent user state are a significant cause of persona drift, and an explicit COMMIT/DEFER/IGNORE routing decision over belief updates addresses it.
- CORE improves response-level personalized alignment on ALOE (82.94 AL) and PersonaChat (76.54 AL) over matched Llama-3.2-3B baselines including RLPA, MemoryBank, and DPO.
- On the held-out PERSIST-Eval benchmark, CORE improves stress robustness versus RLPA (ARUS 80.8 vs 71.9, BCD 76.4 vs 66.1, drift 10.1 vs 12.6) with the largest gains on ambiguity- and conflict-positive turns.
- Structure-matched controls show the gains require executable belief revision, not merely structured traces, action-name supervision, or extra labels (82.9/82.7 vs 80.0/72.1 AL/CRR).
- The mechanism is cheap (23.0 ms/turn overhead) and transfers across 3B backbones (Qwen2.5-3B: 79.6 ALOE AL, 77.4 ARUS).
Strengths
Update-control formulation with structure-matched controls
Beyond adding structured supervision, Table 13 (§E.1) isolates 'action prediction without executable belief revision' (80.0 AL / 72.1 CRR) and 'belief state, no executable action' (80.2 AL / 70.5 CRR) against full CORE (82.9 / 82.7), directly addressing the strongest alternative explanation that gains come from intermediate traces or extra labels.
Metric-source separation and evaluator auditing
Appendix C.1 restricts the GPT-5.1 judge to response-level metrics (AL, ARUS) while belief/action metrics use reference annotations or logged actions; judge-human agreement is reported (J-H 0.78-0.86, κ 0.69-0.82, Tables 7-8) and §C.8 audits collapse to always-DEFER/IGNORE and over-clarification.
Honest scoping of state metrics
Boundary diagnostics beyond the benchmark's comfort zone
Mechanism cost and transfer
Weaknesses
Judge dependence with circularity risk
Thin visible statistical reporting
AL metric construction stretches ordinal data
Scale generality untested with trained comparisons
Label noise propagates into supervision and diagnostics
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper targets **persona drift** — the gradual loss of grounded user preferences in long multi-turn dialogue — and reframes it as a *state-tracking* problem: a turn's local observations must not be written into the persistent user state unless they are sufficiently grounded, resolved, and consistent. Contributions: (1) an update-control formulation of personalization; (2) **CORE**, a slot-factorized belief tracker with COMMIT/DEFER/IGNORE routing and gated exponential belief revision (Eqs. 11-15), trained by SFT warm-up plus PPO (reward weights 0.35/0.30/0.25/0.10); (3) **PERSIST**, a held-out post-anchor stress benchmark (anchoring → fluctuation → confrontation, six social-influence strategies; 1,200 eval dialogues). Headline claims: 82.94/76.54 AL on ALOE/PersonaChat (Table 1, vs RLPA 75.84/65.05); PERSIST ARUS 80.8 vs 71.9 and drift 10.1 vs 12.6 vs RLPA (Table 2); ablations attributing gains to executable belief revision.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is persona drift in LLM dialogue?
Persona drift is the failure where a language model gradually departs from a user's established preferences as a conversation grows — for example, overwriting a grounded preference ('I'm vegetarian') because of a later ambiguous or socially pressured remark. This paper treats premature writes of weak, transient, or conflicting turn-level evidence into the persistent user state as a key cause.
How does the CORE method work?
CORE (Controlled Observation Routing for Evidence-grounded Persona Revision) keeps a per-slot probability distribution over user attributes, with an UNK value for unresolved preferences. Each turn it scores slot-level evidence for relevance, ambiguity, and conflict, then routes it as COMMIT (gated update of the persistent belief), DEFER (relevant but unresolved — no persistent write), or IGNORE (irrelevant or explicitly non-persistent). It is trained with supervised warm-up plus PPO on a Llama-3.2-3B-Instruct backbone and needs no gold labels at inference.
What is the PERSIST benchmark and how is it built?
PERSIST is a held-out synthetic benchmark for post-anchor persona robustness. Each dialogue first anchors persona axioms, then introduces fluctuating partial cues, then applies six social-influence pressures (authority, conformity, affective appeal, urgency, reciprocity, relationship pressure). Systems are scored on preserving grounded commitments (ARUS), resisting blind compliance (BCD), and clarifying under uncertainty (APR), from final responses only — no internal state required. The paper's version spans 1,200 held-out dialogues (15,614 turns) generated and audited via an LLM pipeline with human checks.
Does CORE require gold persona labels or update-action annotations at inference time?
No. According to Appendix B.2 and §4.5, gold normalized persona states and update-action labels are used only for training supervision and evaluation. At inference, CORE operates from the dialogue history, its own maintained belief, predicted turn-local evidence, and predicted routing decisions, adding about 23 ms per turn on the same backbone.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The excerpt contains four '[...sampling gap...]' regions (end of §3.5 into §4.1, end of §4.5 through the conclusion, Appendix C.3-C.6, and Appendix D.6); significance-test details, metric aggregation (C.6), and the full PERSIST evaluation protocol may exist in the full paper, so 'no visible p-values' is a statement about the provided text only.
- No public repository URL is visible in the excerpt; code availability is inferred solely from Appendix D.3's statement that 'accompanying materials include scripts' for benchmark construction and metric computation.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.