Agentic Context Cracking: Token-Efficient Data Reasoning Agents via Adaptive Structuring of Unstructured Data
Milad Rezaei Hajidehi, Qitong Wang, Stratos Idreos
cs.AI · cs.CL · cs.DB
Quick verdict
Proposes agentic context cracking: agents speculatively structure opened docs; 53% cost cut on augmented FanOutQA at preserved accuracy; promising, eval thin.
Summary
The paper addresses the high cost of LLM-agent 'data reasoning': multi-hop questions answered by repeatedly opening large unstructured documents, consuming up to ~1M tokens and nearly $1 per question on FanOutQA with Claude Haiku 4.5. Motivated by a measurement that an ideal hand-built structured store is 28x cheaper on ten sampled questions, the authors propose agentic context cracking: whenever the reasoning agent opens a document, a cracking sub-agent forks from the already-prefilled context (reusing KV-cache/prompt-cache prefixes) and, under a 4K-token decode budget, extracts grounded RDF-style facts with evidence, cardinality and units for future related queries. A catalogue over the stored 'cracked objects' lets later queries answer via compact structured reads, falling back to raw documents on misses; documents are never opened solely for cracking. Because FanOutQA questions lack cross-question locality, the authors augment it with one LLM-generated, human-verified related question per test question and report metrics only on original questions. Results: mean cost falls from $0.26 to $0.12 (53% cut) with LLM-judged accuracy 43.0% to 42.0% (p=0.39) and string accuracy 76.1% to 74.2%; a Hitchcock case study shows a 3x cut. Cracking adds ~12% per-question overhead. Limitations: single model and benchmark, manufactured reuse structure, no ablation against request-only extraction or memory/graph-RAG baselines, and no audit of cracked-object correctness.
Review complete
Assessment at a glance
6
risk flags identified
● high 0
● medium 3
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- On FanOutQA, agentic data reasoning consumes up to ~1M tokens (~$1) per question with Claude-Haiku-4.5, while reasoning over an ideal pre-structured store is 28x cheaper, a gap that widens with fan-out.
- Agentic context cracking cuts mean per-question cost by 53% ($0.26 to $0.12) on FanOutQA augmented with one generated related question per test question, while preserving accuracy (LLM-judge 43.0% to 42.0%, p=0.39; string 76.1% to 74.2%).
- In a Hitchcock investigation with twenty warming questions, cracking cuts cost 3x ($0.81 to $0.27), and cracking pays off on about three quarters of questions (9x cheaper at the 10th percentile, 1.24x baseline cost at the 90th).
- Cracking overhead is bounded and minor: a 4K-token decode budget per question producing roughly 150 cracked objects, about 12% per-question overhead including billed cache reads.
- Cracked objects are evidence-backed plain text, hence a durable, model-agnostic asset unlike model-bound KV caches or semantic answer caches.
Strengths
Novel synthesis of database cracking and agent serving
Transplants the query-driven, pay-as-you-go philosophy of database cracking (Idreos et al., CIDR'07) and dataspaces (iTrails) into LLM agent context management, with a concrete read/write protocol (Fig. 2) whose rule 'never open a document solely to crack' is a clean, principled systems design.
Honest overhead and worst-case accounting
The 12% per-question cracking overhead is stated separately, including vendor cache-read billing (Sec. 4); the CDF analysis (Fig. 3d) openly reports that 90th-percentile questions cost 1.24x baseline with zero reuse — rare candor about non-uniform benefit in this area.
Contamination-aware evaluation design
Accuracy preservation is architectural, not incidental
Cracked objects outlive KV caches
Weaknesses
The reuse structure the method needs is manufactured
Pivotal ablation missing
Cracked-object quality never audited
Thin statistical evidence, tiny case study
Single model, likely-seen corpus
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper targets the dominant cost driver of LLM-agent "data reasoning": multi-hop questions answered by repeatedly opening large unstructured documents. On FanOutQA with Claude-Haiku-4.5, a single question can consume up to ~1M tokens and cost nearly $1 (Sec. 1-2). The motivating measurement (Fig. 1 right): for ten sampled questions whose required facts were manually extracted into a database, answering from that ideal store is 28x cheaper. The proposal, agentic context cracking (Acc), exploits this gap without pre-structuring everything: whenever the reasoning agent opens a document, a cracking sub-agent forks from the already-prefilled context (shared-prefix KV reuse or vendor prompt caching) and, under a 4K-token decode budget, emits schema-constrained "cracked objects" — RDF-style edges carrying subject, relation, object, cardinality, unit, and evidence. A per-document catalogue (Sec. 3) is surfaced before document opens; later queries use constrained structured reads and fall back to raw documents on miss. Documents are never opened solely for cracking.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is agentic context cracking?
A technique for cutting LLM agent costs: whenever a reasoning agent opens a document to answer a question, a small sub-agent reuses the already-loaded context and extracts reusable structured facts (entities, attributes, relations with source evidence) into a store. Later related questions can be answered from these 'cracked objects' without reopening documents, falling back to the raw document when the store misses.
How much does it save?
On the FanOutQA benchmark extended with one related question per test question, mean cost per question fell from $0.26 to $0.12 (a 53% cut) with answer accuracy essentially unchanged; in a 30-question Alfred Hitchcock investigation, cost fell 3x ($0.81 to $0.27). The cracking step itself adds about 12% overhead per question.
Does cracking reduce answer accuracy?
Barely in this study: LLM-judged accuracy went from 43.0% to 42.0% (not statistically significant, p=0.39) and word-level string accuracy from 76.1% to 74.2%, which the authors attribute mostly to answer formatting (canonicalization). The system always falls back to reading the original document when the structured store lacks the needed facts.
How is this different from caching?
Answer caches help only when the same or a near-duplicate question returns, and KV caches are bound to a specific model. Cracking stores evidence-backed facts as plain text, so distinct but related questions can reuse them, and the store survives model upgrades and transfers between LLMs.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure graphics are not present in the extracted text; all quantitative results cited in this review come from the figure captions and body text, which do contain the numbers (e.g., Fig. 3 caption values).
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.