MEMO: Multimodal Evidence Memory Organization for Long-Horizon LLM Agents
Xian Gao, Jinpeng Wang, Jiacheng Ruan, Guangyu Cao, Ting Liu, Yuzhuo Fu
cs.CL
Quick verdict
Trains evidence selection and text/visual layout planning for agent memory readout; clear budgeted QA gains, but no significance tests or code release.
Summary
MEMO is a memory-readout method for long-horizon LLM agents that decides, per evidence unit, whether query-relevant memory is presented to the reader as text, as a rendered visual page, as both, or dropped, under a shared token budget. A fine-tuned evidence extractor (Qwen2.5-1.5B) selects source blocks and character-level spans; a fine-tuned memory manager assigns modality and layout (cards, timelines, tables, checklists); a deterministic builder assembles a text packet plus a visual page. Both components are supervised on training splits of the four evaluation benchmarks, with manager labels derived counterfactually from a frozen reader's answer quality. On HotpotQA, 2WikiMultiHopQA, LoCoMo, and ALFWorld with three frozen readers (InternVL3.5-8B, Qwen3-VL-32B, gpt-5.4-mini), MEMO achieves the best Overall EM/F1 under a strict 128-token budget — e.g., 73.91 F1 on 2Wiki with Qwen3-VL-32B versus 56.26 text-only and 35.89 visual-only — and without a budget uses 83.93 tokens on average (roughly half of MemAgent) while ranking first or second overall. Ablations show the extractor contributes most of the gain. Limitations: no significance testing or variance reporting; per-task wins are uneven (AgentOCR leads some LoCoMo scores); ALFWorld scores are near floor for all methods; the LoCoMo test set is two conversations; and no code release is stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Jointly deciding evidence selection, modality, and layout at evidence-unit granularity under one budget outperforms text-only and visual-only memory readout.
- Under a strict 128-token budget, MEMO achieves the best Overall EM and F1 with all three frozen readers, e.g., 73.91 F1 on 2Wiki with Qwen3-VL-32B versus 56.26 text-only.
- Without a token budget, MEMO averages 83.93 memory tokens — roughly half of MemAgent — while ranking first or second on every Overall metric.
- Ablations attribute most of the gain to evidence extraction (+6.07 EM, -44.4% tokens), with the memory manager adding a further +4.22 EM at modest token cost.
Strengths
Well-posed unified readout formulation
Formulates memory readout as query-conditioned working-memory construction under one shared budget (Ctext + Cvis <= B), jointly deciding evidence selection, modality (text/image/dual/drop), and layout at evidence-unit granularity; the dual-channel action is a sensible middle ground between textual fidelity and visual structure.
Consistent budgeted gains across readers
Table 1: best Overall EM/F1 under the strict 128-token budget with all three readers; flagship cell is 2Wiki F1 73.91 (MEMO) vs 56.26 (Text-only) and 35.89 (Visual-only) with Qwen3-VL-32B; InternVL3.5-8B margin over the best baseline is +7.30 EM / +7.22 F1.
Efficiency without quality collapse
Clean component ablation
Honest per-task reporting
Weaknesses
No statistical rigor
In-domain oracle supervision advantage
Training-reader affinity confound
ALFWorld protocol is near-floor
Reproducibility gaps and token accounting
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
MEMO (arXiv 2609.07471) targets the read side of agent memory: given a query, candidate memory records, and a token budget, construct the working memory handed to a frozen VLM reader. The pipeline is (1) a trained evidence extractor (Qwen2.5-1.5B-Instruct) that selects source blocks and character-level key spans; (2) a trained query-conditioned memory manager that assigns each evidence unit an action from {text, image, dual, drop} plus confidence, budget target, and visual-template preference; (3) a deterministic builder that assembles a text packet T and a rendered visual page V with Ctext + Cvis <= B. Headline claims: best Overall EM/F1 under a strict 128-token budget across three readers; near-best quality without a budget at 83.93 average tokens; and distinct contributions from both trained components.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does MEMO do in one sentence?
MEMO is a memory-readout method for LLM agents that, given a query and a token budget, selects relevant memory evidence and decides for each piece whether to show it to the reader model as text, as a rendered visual page (cards, timelines, tables, checklists), as both, or to drop it.
How much better is MEMO than plain text memory?
Under a strict 128-token memory budget with Qwen3-VL-32B as the reader, MEMO scores 73.91 F1 on 2WikiMultiHopQA versus 56.26 for presenting memory as plain text and 35.89 for a purely visual page; it also achieves the best overall EM/F1 across all three tested readers under that budget.
Does MEMO require training, and on what data?
Yes. Two Qwen2.5-1.5B models are fine-tuned (an evidence extractor and a memory manager) on the training splits of the four evaluation benchmarks; the manager's labels come from trying counterfactual presentation choices and scoring answers with a frozen InternVL3.5-4B reader.
Is MEMO's code available?
The paper does not mention any code or data release, and key details such as hyperparameters and the exact visual-token accounting are not specified, so independent reproduction currently requires re-implementing the full training and rendering pipeline.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- PDF extraction is complete through the reference list (ends at Zou et al., arXiv:2601.07582); no truncation detected.
- Figure interiors (Figures 1-2) are not recoverable from text extraction; the workflow was judged from captions and body text only.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.