SpeakerMem-R1: Speaker-Centered Dual-Track Memory for Multi-Party Dialogue
Haobo Zheng, Tan Tang, Yan Chen, Weijie Wang, Yingcai Wu
cs.CL · cs.AI · cs.IR · cs.LG
Quick verdict
Dual-track speaker-attributed memory + RL-trained 3B writer shows solid multi-party gains; thin RL data, config selection and weak LoCoMo transfer temper it.
Summary
SpeakerMem-R1 targets long-term memory for multi-party group chats, where generic LLM memory systems lose speaker attribution, person/group scope, and evolving states. The system writes a dual-track memory: System 1 stores verbatim speaker-labeled messages deterministically; System 2 stores derived records in four layers (person-level Core/Profile, group-level Interaction/Insight) with source/owner fields and provenance links, updated non-destructively via ADD/UPDATE/NOOP chains. At query time, Project compiles the question and roster into constraints, and Anchor-Separate-Resolve-Compose organizes evidence from both tracks for a frozen answerer. A Qwen2.5-3B Writer is trained with SFT and then reinforcement learning (SpeakerLevenshtein owner-level writing reward plus speaker-conditioned GRPO). Reported binary accuracies are 47.9% (GroupMemBench), 69.2% (SocialMemBench), and 61.9% (EverMemBench), 3.3-12.4 points above the best mainstream baselines (including BM25 and A-MEM); it reports 62.33% (1,496/2,400) on the public EverMemBench leaderboard and 70.85% on LoCoMo as a two-person boundary test. On 305 held-out SocialMem questions, RL lifts the SFT writer from 57.38% to 68.20% (mean±std over 3 seeds), reaching 95.4% of a DeepSeek-V4-Flash LLM writer (71.48%). Ablations confirm both tracks and both hierarchy levels contribute. Limitations: per-benchmark selection of the best configuration, a tiny RL training corpus (15 networks; only 3 real UPDATE actions), LLM-judge metrics without agreement analysis, and weak LoCoMo multi-hop/open-domain results.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 0 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A dual-track memory separating verbatim speaker-labeled messages from person- and group-level derived states, with source-owner provenance, targets the attribution and state-reconstruction failures of general memory systems in multi-party dialogue.
- SpeakerMem-R1 reaches 47.9%/69.2%/61.9% binary accuracy on GroupMemBench/SocialMemBench/EverMemBench, exceeding the best mainstream baselines by 3.3/12.4/9.4 points respectively.
- On the public EverMemBench leaderboard it reports 62.33% (1,496/2,400), the best published result, ahead of EverOS at approximately 60.08%.
- Speaker-conditioned GRPO with a SpeakerLevenshtein reward lifts a Qwen2.5-3B Writer from 57.38% to 68.20% on 305 held-out questions with the query/answer pipeline frozen, reaching 95.4% of a DeepSeek-V4-Flash LLM writer.
- Ablations show both tracks and both hierarchy levels are complementary: removing the verbatim track costs 14.75-26.6 points across the three benchmarks.
Strengths
Problem formalization connects failure modes to mechanisms
The record schema r=(x, src, own, scope, event, time, state, ref) with source/owner separation and PERSON/GROUP scope directly targets conflation errors; the five diagnostic dimensions (Appendix F.1, Tables 14-15) map observed failure modes to specific mechanisms rather than inventing an unvalidated taxonomy.
Controlled Writer experiment is methodologically clean
Table 3: frozen query/answer pipeline, matched no-ASK protocol, three seeds with sample std — RL gains 57.38±0.33% to 68.20±0.66% (+10.82pp), with the honest -3.28pp gap to the DeepSeek-V4-Flash LLM writer (71.48±0.66%) and an explicit disclaimer against cross-domain generalization claims.
Comprehensive ablation coverage
Unusual transparency for reproduction
Honest negative results
Weaknesses
Per-benchmark configuration selection in headline results
Tiny, proxy-supervised RL corpus with almost no real UPDATE signal
No statistical testing for the main benchmark comparisons
EverMem leaderboard comparison changes configuration and judge
Boundary test exposes two-person weaknesses
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 0 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is SpeakerMem-R1?
A long-term memory system for multi-party group chats built around LLM agents. It stores two tracks: verbatim messages labeled with speaker, time and channel, plus structured derived states organized per person and per group, with fields separating who stated a fact from whom it concerns. At question time it retrieves evidence from both tracks and hands it to a frozen answering model. Its Writer component is a Qwen2.5-3B model trained with SFT plus reinforcement learning.
How well does SpeakerMem-R1 perform?
The paper reports binary accuracies of 47.9% on GroupMemBench, 69.2% on SocialMemBench, and 61.9% on EverMemBench — 3.3 to 12.4 points above the best mainstream baselines it compares against (including BM25 and A-MEM). On the public EverMemBench leaderboard it reports 62.33% (1,496 of 2,400 questions), ahead of EverOS at about 60.08%. On LoCoMo, used as a two-person boundary test, it reaches 70.85% overall but trails graph-RAG methods on multi-hop and open-domain questions.
What is the SpeakerLevenshtein reward?
It is the paper's owner-level writing reward for training its memory Writer: an edit-distance-style score that measures how accurately structured memory writes preserve who said what and whom each statement concerns, combined with a terminal question-answering gain inside a speaker-conditioned GRPO objective. The formal definition appears in the paper's training section (partially truncated in this extraction); Appendix E documents the GRPO mechanics, training networks and hyperparameters.
Is code or data available for SpeakerMem-R1?
The paper lists a project page (2022hpsk.github.io/SpeakerMemR1) and a GitHub repository (github.com/2022hpsk/SpeakerMemR1). Appendices provide the full memory schema, all key prompts verbatim, training hyperparameters, and a reproducible training-network inventory. LoCoMo is a public benchmark; the three multi-party benchmarks come from very recent 2026 papers whose availability was not verifiable in this review.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Section 4 (Writer training, including the SpeakerLevenshtein reward definition) and parts of Section 5 and Appendices C-E fall in truncated extraction regions; assessment of the RL objective relies on Appendix E fragments.
- Figures 1-6 appear only as embedded text labels; ablation values were read from caption text and could be mis-assigned across variants.
- The three cited multi-party benchmarks (Yang 2026; Owolabi 2026; Hu 2026b) and baseline configurations could not be independently checked; baseline numbers rely solely on this paper's description.
- arXiv v2; no code audit or external replication was performed (network access disabled for this review).
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.