What Attention Recalls and Recurrence Controls in Hybrid Language Models

Summary

The paper asks what the attention and recurrent channels each contribute in hybrid language models. The authors introduce two cache-level interventions: split-prefill, which prefills a context and keeps only the KV cache or only the recurrent state when answering a query, and state-swap, which pairs the KV cache of one context with the recurrent state of another. On four small synthetic tasks (dictionary retrieval, list indexing, language following, persona style) across Qwen3.5 (4B-27B), Falcon-H1 (3B-34B), and three further hybrids, exact retrieval survives only through the KV cache (64-98% of full accuracy on the main models; recurrent-only collapses to ~0), while language following and persona survive only through the recurrent state (70-80% language; KV-only ~1%). State-swap shows causally that answers take their content from the KV side and their language from the recurrent side. Two further traps (DRM-style associative false recall and memory-conjunction errors) show recurrent-only generation accepts unseen words sharing meaning or morphemes with seen items. The authors are candid that interventions create partly out-of-distribution channel combinations and that claims are functional, not static. Main limitations: results rest on small synthetic tasks; the persona dissociation reverses in two larger Falcon-H1 configurations (7B and 34B, marked in Table 2); and persona scoring depends on a single LLM judge.