StateComp: Learning When to Compress History in Long Horizon Agents
Mingxuan Wang, Hongyue Chen, Yinglong Guo, Yinglong Guo, Fei Luo, Chao Ning, Chao Ning, Bo Wang, et al.
cs.AI
Quick verdict
StateComp cuts agent tokens 52.27% at near-flat reward via state-based compression timing; promising, but single-run evals lack statistical rigor.
Summary
StateComp addresses context management for long-horizon LLM agents by asking when each past interaction becomes safe to compress, rather than how to shorten a prompt. The method labels (interaction, current-state) pairs as KEEP or READY via two-stage LLM annotation: Stage 1 judges each checkpoint using only prefix evidence (future steps may only veto), and Stage 2 selects the earliest supported compression boundary per interaction. An imbalance-aware router trained on frozen Qwen2.5-7B hidden states (AUROC 0.7096, PR AUC 0.1736) predicts readiness online; adjacent READY interactions form spans that are summarized only when τ=0.60, at least 4 interactions, and at least 1,000 source tokens are met, and committed summaries enter the effective history. A bounded 5,120-token state representation yields a 12.67× representation-extraction speedup. On WorkBuddyBench (260 tasks, DeepSeek-V4-Flash), total tokens fall 52.27% (698.17M to 333.24M) with mean reward essentially flat (0.6987 to 0.7026); on Eval40, StateComp achieves the best reward and token combination among thirteen context-management baselines, and the pattern holds across four acting backbones. Main limitations: no statistical tests or run-to-run variance for small reward deltas; LLM-generated supervision without quantified human validation; router operating points show very low recall (72.22% precision at 0.11% recall); the router study uses Qwen2.5-7B while the online controller uses Qwen3.5-9B; no code release is stated.
Review complete
Assessment at a glance
6
risk flags identified
● high 1
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Compression timing is a state-dependent property of (interaction, current state) pairs, and two-stage annotation with prefix-only evidence and future veto yields supported KEEP/READY boundaries.
- Frozen hidden states are partially predictive of compression readiness, reaching AUROC 0.7096 and PR AUC 0.1736 under grouped cross-trajectory evaluation.
- StateComp reduces total agent and summarization tokens by 52.27% on the 260-task WorkBuddyBench (698.17M to 333.24M) while maintaining mean reward (0.6987 to 0.7026).
- On Eval40, StateComp achieves the highest mean reward (0.7154) at 43.76M tokens, beating thirteen published context-management baselines on both axes.
- A bounded 5,120-token state representation speeds representation extraction 12.67x (76.40s to 6.03s on seven paired states) without lowering test F1 (0.7818 vs 0.7000).
Strengths
Clear and novel problem framing: compression timing as a state-dependent (interaction, state) decision
§3.1–3.2 and Eq. 1 define per-pair labels y_i,k and a monotone earliest-boundary encoding, cleanly separating 'when can this be removed' from conventional relevance selection — a distinction missing from fixed-window and periodic methods
Disciplined anti-leakage annotation design (§3.3, A.2)
Stage 1 requires positive evidence from the prefix with future steps restricted to vetoes (Eq. A.3); Stage 2 selects the earliest supported boundary, and unresolved cases default to KEEP — this blocks retrospective label leakage
Prediction/execution separation demonstrably controls overhead (§4.2, Table A9)
Unusually candid self-reporting
Transfer demonstrated across four acting models with fixed task IDs (Table 4, A12)
Weaknesses
No statistical treatment of task-level results
LLM-authored supervision without human validation (§3.3, A.3, AI Use Statement)
Router's marginal value online is unproven (§5.2, §4.2)
Backbone mismatch between study and deployment (§5.1 vs E.1)
Router cost scales with history length and timing evidence is thin (C.1–C.3, A3)
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
StateComp targets context management for long-horizon LLM agents. Its central reframing: the question is not *how* to shrink a prompt but *when* each past interaction has become safe to replace, a decision that depends on the current agent state. The paper claims (1) a state-dependent supervision scheme built on a two-stage annotation that finds the earliest supported compression boundary t*_i per interaction (Eq. 1); (2) an imbalance-aware router on frozen hidden states (AUROC 0.7096, PR AUC 0.1736, §5.2); (3) a 52.27% token reduction at maintained reward on WorkBuddyBench (Tables 2, A6–A7); (4) dominance over thirteen published baselines on Eval40 (Table 3); and (5) a 12.67× representation-extraction speedup via bounded 5,120-token state views (Table 1). Evidence rests entirely on one benchmark family (Tencent WorkBuddyBench) and API-backed acting models.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is StateComp and what problem does it solve?
StateComp is a framework for long-horizon LLM agents that decides, at each execution step, which past interactions have become safe to replace with summaries given the current agent state. It trains a router on frozen LM hidden states and only rewrites spans of at least 4 interactions and 1,000 source tokens. On WorkBuddyBench it cuts total agent and summarization tokens by 52.27% while keeping average task reward essentially unchanged.
How many tokens does StateComp save?
On the full 260-task WorkBuddyBench run, total tokens fell from 698.17M to 333.24M (−52.27%). On the 40-task subset (Eval40), tokens fell from 75.50M to 43.76M (−42.04%), of which only 0.077M were summarization overhead.
Does compressing history hurt agent task performance?
Mean reward was essentially flat on the full benchmark (0.6987 baseline vs 0.7026 with StateComp, on a 0–1 scale), and slightly higher on Eval40 (0.6754 vs 0.7154). However, the paper reports single runs without statistical significance testing or variance, so small reward differences should be interpreted cautiously.
Is code or data available for StateComp?
No. The paper states that appendices document the annotation procedure, evaluation splits, hyperparameters, and fixed Eval40 task identifiers, but it does not mention any code, data, or model-weight release.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures (Fig. 3, Fig. 4, A1–A5) are available only as captions and axis labels; trend claims rely on the surrounding text.
- Exact annotation prompts, annotator model identifiers, and sampling temperatures (Appendix A/B details) are not visible in the extracted text.
- Several cited works are 2026 arXiv/blog items (WorkBuddyBench, DeepSeek-V4-Flash, Qwen3.8-Flash, SelfCompact) that could not be verified against external sources.
- Absence of a code/data release is judged from the extracted text only; no reproducibility URL appears anywhere in it.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.