Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents
Yefan Zhou, Yang Li, Zeyu Leo Liu, Semih Yavuz, Shafiq Joty
cs.AI
Quick verdict
JITMEM defers memory curation to read time, trains curator with GRPO; big gains on ALFWorld/WebShop, modest on τ²-bench; good ablations, no significance tests.
Summary
The paper argues agentic memory should be curated at read time rather than write time: distilling a trajectory into a fixed artifact before the future query is known causes irreversible information loss and yields query-independent summaries. JITMEM instead stores raw trajectories (only those an executor-as-judge deems successful), retrieves the top-3 via BM25 over task descriptions, and uses a curator LLM to synthesize a compact, task-adaptive payload for the current task. Because the payload is consumed immediately, the curator is trained with GRPO against same-task reward with a frozen executor, collapsing credit assignment to one step. On ALFWorld, WebShop, and τ²-bench, JITMEM improves absolute success rate over the strongest baseline by 16.2, 16.3, and 3.9 points respectively; the untrained curator is already competitive with most write-time baselines, and the trained curator transfers across executors (within 1.4 points of a directly trained one). Payloads cut input tokens roughly 50–56% and executor steps roughly 28–31% versus write-time methods. Ablations attribute gains to task-conditioning, success filtering, and raw storage. Main limitations: no significance tests, gains shrink with stronger executors and are absent on two of three τ² domains, judge-based storage gating is unvalidated, and no code release is stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Deferring memory curation to read time makes distillation task-adaptive: the same stored trajectory yields different payloads for different queries.
- Because the payload is consumed on the same task, the curator can be trained with GRPO on immediate task reward, eliminating the delayed-credit and task-grouping machinery of learned write-time curators like SkillOS.
- JITMEM improves absolute success rate over the strongest baseline by 16.2 (ALFWorld), 16.3 (WebShop), and 3.9 (τ²-bench micro) points under matched configurations.
- Even an untrained read-time curator is competitive with or surpasses write-time baselines using the same curator model, and the trained curator transfers across executors within 1.4 SR of a dedicated one.
- Ablations attribute the gains to task-conditioned curation, success-filtered storage, and raw-trajectory retention, with RL gains dependent on actual retrieval (up to −15.2 SR without retrieved trajectories).
Strengths
The right control experiment for the central claim
The 'w/o retrieved traj.' ablation (Sec 4.2, Table 9) forces an empty retrieval set and shows the RL-trained curator drops by up to 14.8 SR on ALFWorld and 15.2 on WebShop — directly ruling out the alternative explanation that GRPO merely taught the curator to emit useful hints from parametric knowledge.
Matched-capacity comparisons isolate curation strategy from curator strength
Sec 4.1 compares training-free variants under the same curator model: with Qwen3-8B as both curator and executor on ALFWorld, JITMEM-base reaches 60.5 SR vs 55.7 (ReasoningBank) and 53.1 (SkillOS-base); with Gemini-2.5-Pro on WebShop, JITMEM-gemini hits 61.0 SR vs 41.0 (SkillOS-gemini). The 'weaker curator beats stronger-curator baselines' result (79.3 JITMEM-base/Qwen vs 77.9 ReasoningBank/GPT-5.4 on GPT-5.4 executor) reinforces this.
Efficiency and transfer are quantified, not just accuracy
Unusually thorough setup disclosure
Weaknesses
No inferential statistics anywhere
Gains shrink with executor strength and vanish on two of three τ² domains
LLM-as-judge storage gating is unvalidated
Baseline provenance and prompt asymmetries
The 'learning' half of the contribution is untested on τ²-bench and no code is released
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper addresses a design question in LLM-agent memory: *when* in the agent lifecycle should experience be shaped? It argues that write-time curation (Reflexion-style reflections, ReasoningBank strategies, MemP multi-granularity items, RL-trained SkillOS) commits to a query-independent artifact before the future task is known, causing irreversible information loss, and that learning such curators suffers a delayed credit-assignment problem. JITMEM stores raw trajectories losslessly, retrieves top-3 by BM25 over task descriptions, and synthesizes a task-conditioned payload at read time using a curator trained with GRPO against immediate task reward with a frozen executor (Sec 3, Figure 1). Four claims are made: (1) read-time curation is task-adaptive; (2) it collapses credit assignment to one step, removing SkillOS's task-grouping scaffolds; (3) JITMEM beats the strongest baseline by +16.2/+16.3/+3.9 absolute SR on ALFWorld/WebShop/τ²-bench; (4) even the untrained curator is competitive. Within the evaluated scope these are mostly supported — but scope excludes trained variants on τ²-bench (no training split) and any large-bank regime.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the difference between write-time and read-time memory curation for LLM agents?
Write-time systems (Reflexion, ReasoningBank, SkillOS) distill each finished task into a fixed artifact (reflection, strategy, skill) that is stored and later retrieved by similarity. Read-time curation, as in this paper's JITMEM, stores raw trajectories untouched and only distills them into a summary when a new task arrives, so the distillation can be tailored to that specific task instead of being a one-size-fits-all artifact fixed before the query is known.
How much does JITMEM improve over prior memory methods?
On the paper's three benchmarks it improves absolute success rate over the strongest baseline by 16.2 points on ALFWorld (77.4 vs 61.2 for SkillOS), 16.3 points on WebShop (32.8 vs 16.5), and 3.9 points on τ²-bench micro average (75.6 vs 71.7 for ReasoningBank with GPT-5.4 executor). Gains are largest with the weakest executor (Qwen3-8B) and smallest on τ²-bench, where two of three domains show no improvement over a no-memory agent.
Does JITMEM require training, and what does training cost?
No — the untrained curator is already competitive with most write-time baselines. RL training with GRPO (100 steps, Qwen3-8B curator, frozen executor) compounds the gains; it takes about 21 hours on ALFWorld and 27 hours on WebShop on a single 8×H200 server. The trained curator transfers to stronger executors (GPT-5.4, Gemini-2.5-Pro) without retraining, within 1.4 SR points of a curator trained directly on the target executor.
Is code or data available for JITMEM?
The preprint text reviewed here contains no code or artifact release statement. It does publish all curator, executor, and judge prompts (Appendix A) and full GRPO hyperparameters (Table 7), and all three benchmarks (ALFWorld, WebShop, τ²-bench) are public — but reproducing the trained variant needs frontier API access plus roughly a day on 8×H200 GPUs.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The abstract is truncated mid-sentence in the extracted text ('even an u...'); its final sentence was inferred from the body.
- Figures 1-4 are represented only by captions and partial embedded text; visual details of the ablation bar charts (Figure 2) could not be independently verified.
- Only the printed table values were used; no external verification of baseline implementations (ReasoningBank, MemP, SkillOS) was possible.
- No statement about code release was found in the provided text; if the arXiv page or appendix links to code, that would change codeAvailable.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.