Machine-Interpretable Information: Compiling Documents into Searchable and Readable Protocol States
cs.CL · cs.LG
Quick verdict
Compiles docs into 56-token states readable by frozen LLMs; Residual-MII beats full-context EM on HotpotQA at ~7% FLOPs, but F1 lags and evidence is thin.
Summary
The paper introduces Machine-Interpretable Information (MII), a protocol that 'compiles' documents into a fixed 56-token continuous state using a dual-timescale Mamba writer (~0.2B params, GPT-2 tokenizer), which per-reader Translators (~0.5B total) project into frozen LLM embedding spaces (Llama-3-8B, Qwen-2.5-7B, Mistral-7B). The state combines three memory banks (global/event/cover) and is packaged as a transferable '.mii' artifact claimed to unify retrieval, reasoning, and reconstruction. On the full HotpotQA dev set (7,405 queries), pure MII scores EM 29.9% vs 19.1% blind; Residual-MII(Dense)—the state plus ~480 tokens of retrieved text—reaches EM 46.7% versus 46.4% for full-context reading at ~7% of attention FLOPs, though F1 is lower (59.6 vs 61.1). The authors' own 1,024-token GPT-5.5 text summary reaches EM 52.0%, exceeding both. Ablations show the global bank is critical, and masking the event bank helps pure MII but hurts Residual-MII. Mechanistic probes include ΔNLL-guided latent transplantation and zero-shot 5-hop reasoning over concatenated states. Key limitations: no significance testing, missing autoencoder baselines promised in Section 3, an ultra-long-context table without baseline numbers, internal numeric inconsistencies, and no code release.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 4
● low 0
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A 56-token compiled protocol state, produced by a dual-timescale Mamba writer and mapped by per-reader Translators, is readable by heterogeneous frozen LLMs (Llama-3-8B, Qwen-2.5-7B, Mistral-7B) despite a forced GPT-2 vocabulary chasm (Table 4).
- Residual-MII(Dense) achieves higher Exact Match than full-context reading on the full HotpotQA dev set (46.68% vs 46.39%, n=7,405) at ≈7% of the attention FLOPs (Table 1).
- The .mii state has modular, causally traceable latent structure: entity tokens can be zero-shot transplanted across unrelated states, and two concatenated states support 5-hop reasoning without joint retraining (§4.5, Appendix G).
- The fixed 56-token bandwidth is not the binding constraint — writer model capacity is, per capacity-scaling reconstruction curves (Appendix F).
- Masking the event bank helps pure MII but hurts Residual-MII, supporting an L1/L2 cache-hierarchy interpretation of the memory banks (Table 3).
Strengths
Well-specified, genuinely novel protocol design
Dual-timescale Mamba writer with three complementary memory banks (16 global / 24 event / 16 cover, §2.1) explicitly targets distinct pooling failure modes; the GPT-2 'vocabulary chasm' (§2) is a clever design that rules out token-ID pass-through as a shortcut.
Honest and informative ablations
Table 3 shows masking the Event bank helps pure MII (+10.4% EM) but hurts Residual-MII (−3.8%), empirically grounding the L1/L2 cache-hierarchy hypothesis; the Global bank's necessity (−99% EM when masked) is cleanly demonstrated on 500 samples.
Unusual implementation transparency for a preprint
Mechanistic probing beyond headline metrics
Amortization economics made concrete
Weaknesses
Headline margin is sub-noise and metric-selective
The paper's own baselines undercut the core narrative
Ultra-long-context claims lack supporting numbers
Internal numeric inconsistencies
No code, weights, or artifacts; corpus unspecified
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper proposes MII, framed as the first agent-to-agent document-to-state protocol: a dual-timescale Mamba Writer (~226M active params, GPT-2 tokenizer) compiles documents into a fixed 56-token state (16-token global attention-pooled bank, 24-token event bank selected by fast-branch variation, 16-token uniform coverage bank), and a per-Reader Translator (linear + 2 encoder layers, ~277M) projects it into frozen LLM embedding spaces. Central claims: (1) cross-model readability (Table 4: best EM 28.4 for Llama-3-8B, 23.2 Mistral, 22.2 Qwen-2.5); (2) Residual-MII(Dense) exceeds full-context EM (46.68% vs 46.39%) at ~7% of attention FLOPs on the full HotpotQA dev set (Table 1); (3) modular, editable latent structure (ΔNLL-guided transplantation, 5-hop reasoning over concatenated states, §4.5); (4) capacity is writer-bound, not bandwidth-bound (Appendix F). The scope is a proof of concept; §6 concedes pure .mii 'struggles to verbatim-reconstruct rare entities'.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Machine-Interpretable Information (MII)?
MII is a proposed 'file format' for LLM memory: a small Mamba-based writer model compiles a whole document into a fixed 56-token continuous state, and a per-model translator maps that state into any frozen LLM's embedding space, so the LLM answers questions from the state instead of re-reading raw text. The paper envisions these '.mii' artifacts being compiled once and exchanged between AI agents.
Does Residual-MII really beat full-context reading on HotpotQA?
Only on Exact Match, and narrowly: 46.68% vs 46.39% on the 7,405-question dev set, at about 7% of the attention FLOPs. On F1 it is lower (59.6 vs 61.1), and the authors' own 1,024-token GPT-5.5 text summary scores a higher EM (52.0%).
How large is a .mii file?
Per the paper's format spec, the payload is 56 tokens × 2048 dimensions in bfloat16 (roughly 229 KB), preceded by a JSON header with metadata, a mean-pooled centroid vector for dense routing, and — in the Residual-MII variant — optional cached text chunks.
Is code or data available for this paper?
No public code, weights, or .mii artifacts are mentioned in the paper. It does provide detailed hyperparameters, evaluation configurations, and the .mii format specification, but the training corpus is described only as 'QA contexts and encyclopedic articles', and the GPT-5.5 summary baseline is not reproducible.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–5 are accessible only through captions; the Pareto-frontier, manifold-visualization, spectral-separation, and scaling-curve claims could not be inspected directly.
- Table 4's evaluation subset and the 500-sample ablation splits (Table 3) are unspecified, limiting independent verification of those numbers.
- The extraction appears complete through Appendix H, but multi-column table rendering may have altered cell layouts; all numbers quoted are as extracted.
- GPT-5.5 and several 2026-dated references could not be verified offline.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.