The Implications of Linguistic Illegibility for LLM Security

Summary

This position paper argues that security mechanisms relying on an LLM's linguistic self-reporting cannot be fully sound, and proposes a systems-level sandbox whose guarantees do not depend on reading the model's language. Mickens coins 'linguistic illegibility' for cases where chain-of-thought traces, constitutional self-critique, or mechanistically probed language features fail to reflect internal computation. Section 2 grounds the claim in empirical evidence (Turpin et al.'s biased-CoT rationalization, Lanham et al.'s CoT-editing perturbations, Claude 3.7 Sonnet verbalizing reward hacks under 2% of the time in 5 of 6 scenarios, Pfau et al.'s dots-only CoT) plus a capacity argument from feature superposition: more features than vocabulary tokens implies some features necessarily resist linguistic description. Section 3 proposes five design principles (Table 1): taint-tracking provenance labeling with causal tags and auditable detainting (three candidate tagging algorithms are sketched but none implemented), robust virtualization via seL4 or TEEs, a default-deny egress proxy, statistical behavioral anomaly detection, and third-party configuration attestation. The paper maps these principles to three 2026 sandbox escapes (the OpenAI-HuggingFace package-manager exploit, Anthropic's Mythos browser-and-container escape, and configuration-error incidents), arguing each would have been mitigated. Main limitations: no implementation, prototype, or quantitative evaluation; the overhead and feasibility of taint tracking across a full inference stack are unquantified; implicit flows are covered only probabilistically.