LatentPress: Context Compression Beyond Text and Vision
cs.LG · cs.AI
Quick verdict
A tiny adapter writes context as soft tokens a frozen LLM reads directly; solid speed and accuracy gains, but headline results use oracle evidence retrieval.
Summary
LatentPress compresses conversational histories and long documents into continuous soft-token vectors that a frozen LLM decoder reads directly through its input-embedding interface, with no text reconstruction at inference. The writer is just two borrowed frozen transformer layers plus a small identity-initialized linear adapter (4.2M-26.2M parameters, roughly 0.1% of the decoder), trained on UltraChat with a reconstruction plus forward-KL distillation objective; per-segment compression rates are hand-specified. In the oracle-evidence setting of LongMemEval (500 questions, retrieval idealized), the role-aware writer reaches 0.476-0.504 accuracy at 4.62-7.70x compression versus 0.490 for uncompressed evidence, beating ICAE (0.452 at 4.12x), DeepSeek-OCR reconstruction (0.426 down to 0.312), and text summaries (0.184); this holds across Qwen2.5-7B, Qwen3-8B, and Qwen3-1.7B readers. On LongBench-QA, in-domain writers trained on target-domain splits exceed raw context at 4x (e.g., 57.99 vs 47.93 for Qwen2.5-14B) but fall below it at 16x on all readers; cross-domain transfer wins only at 4x. Writing takes about 43ms per conversation and reading is 5.0-9.2x faster than raw context. Main limitations: the headline comparison abstracts away retrieval, the margin over uncompressed evidence is within one standard deviation, strong training-free text-pruning baselines such as LLMLingua are missing, and gains depend on the hand-specified role schedule.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A frozen decoder can consume compressed context as continuous soft tokens via its input-embedding interface, with only a ~0.1%-parameter adapter trained and no text reconstruction at inference.
- On LongMemEval oracle-evidence QA, role-aware LatentPress matches uncompressed evidence (0.504 at 7.70x vs 0.490) and beats ICAE, DeepSeek-OCR reconstruction, and text summaries, transferring zero-shot from UltraChat across three Qwen readers.
- On LongBench-QA, in-domain writers match or exceed raw-context accuracy at 4-8x compression on all three readers while 16x trails raw.
- Writing runs at ~43ms per conversation (roughly an order of magnitude faster than OCR or summarization routes) and reading from the compressed prefix is 5.0-9.2x faster than raw-context inference.
Strengths
Genuinely lightweight and well-positioned interface
The frozen-reader, direct input-embedding design with only 4.196M-26.220M trainable parameters (~0.1% of the decoder, Sec. 2.3, App. A.1) is cleanly differentiated from Gist/AutoCompressor/ICAE (LLM-scale training) and xRAG (single-token) in Table 1, and the no-reconstruction claim is consistently maintained across training and inference.
Strong zero-shot transfer at trivial training cost
Trained on only 2,000 UltraChat conversations (400 chunks, 1,000 steps, Table 6) with no QA labels, the role-aware writer transfers to LongMemEval on three readers spanning 4.7x scale (0.476-0.504 on Qwen2.5-7B; 0.506-0.514 on Qwen3-8B; 0.416-0.434 on Qwen3-1.7B, Table 3).
Thorough ablation of where the gains come from
Multi-axis, honestly-caveated efficiency measurement
Complete reproducibility apparatus
Weaknesses
Flagship comparison idealizes retrieval
Key 'matches uncompressed' claims sit within seed noise
Thin baseline coverage on the text side
In-domain wins require target-domain training, and the strongest reader flips the story
The paper's own uniform variant collapses far below ICAE
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
LatentPress targets the representation layer of context compression: a writer maps text segments into continuous vectors that a frozen decoder reads via its input-embedding interface (inputs_embeds), with no text reconstruction at inference (Sec. 2, Eq. 1). The stated contribution is the WRITE/READ interface rather than a learned schedule: rates are hand-specified (role-based k_user=1, k_assistant in {8,16,32} for dialogue; uniform k for documents, Sec. 2.2). Central claims: (1) matching uncompressed-evidence accuracy on LongMemEval at up to 7.70x compression with ~0.1% trainable parameters; (2) in-domain LongBench-QA writers matching or exceeding raw context at 4-8x; (3) order-of-magnitude write speedup (43ms) and 5.0-9.2x read speedup. Scope is explicitly narrowed: retrieval is idealized (oracle evidence sessions, Sec. 3) and memory-system concerns are deferred (Sec. 6).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is LatentPress?
LatentPress is a method that compresses long conversation histories or documents into a short sequence of continuous vectors ('soft tokens') that a frozen LLM reads directly through its input-embedding interface — instead of storing context as text that must be re-read or images that must be OCR-decoded.
How much does LatentPress compress and how accurate is it?
It compresses 4-16x. On LongMemEval with the correct evidence sessions provided (an idealized, oracle-retrieval setting with 500 questions), it reaches 0.504 accuracy at 7.70x compression versus 0.490 for reading the full uncompressed evidence, and it beats ICAE, OCR-reconstruction pipelines, and LLM text summaries.
Does LatentPress require fine-tuning the language model?
No. The decoder stays completely frozen; only a small adapter (4.2M-26.2M parameters, about 0.1% of the model) plus two borrowed frozen transformer layers form the writer, trained on as few as 2,000 generic conversations.
Is LatentPress faster than alternatives?
In the paper's measurements on an H100 GPU: writing takes about 43ms per conversation (roughly 10-20x faster than OCR reconstruction, summarization, or ICAE encoding), and answering from the compressed prefix is 5.0-9.2x faster than reading the raw context.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- PDF extraction flattens inline math: subscripted loss/distribution symbols appear as plain text (e.g., Lrec, Lfkl, pfull), and the per-rate symbol in Tables 4/13/15 renders as 'f 4/f 8/f 16', likely a lost glyph. The text is otherwise complete through Table 16, with no truncation.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.