Permutation-Based Stegomalware in Large Language Models: Threats and Countermeasures
cs.CR · cs.AI · cs.LG
Quick verdict
A clean dual-use result: an information-theoretic stegomalware embedding that is provably behavior-preserving, plus a derangement-based defense reaching 100% of LLM parameters.
Summary
This paper studies permutation symmetry of transformer weights from both the offensive and defensive side. Offensively it introduces PermaNet: because multi-head attention, gated MLPs and RMSNorm are equivariant to a permutation P of the residual-stream axis, an adversary can reorder every weight tensor consistently and leave the computed function mathematically identical, embedding floor(log2 n!) bits in the ordering of rows and columns. A non-lexicographic linear-time ranking algorithm is adapted to a non-recursive form so large matrices can be handled. Capacity is substantial: 173 KB for TinyLlama, 714 KB for Mistral-7B and Llama-3-8B, and 2.78 MB for GPT-OSS-20B, mostly from per-expert MLP inner dimensions. Because the decoder recovers a rank rather than bytes, it is payload agnostic and no retraining is needed. Defensively, the same symmetry yields full neutralization: applying random derangements across embedding, expert, KV-head and MLP inner dimensions moves 100% of parameters, against 60% for NeuPerm, and raises neutralization from 0% to 100% on embeddings and from 77% to 100% on attention query matrices for MaleficNet and LSB payloads. Finite precision is the only cost: on TinyLlama bf16 the resulting KL divergence is 5.8e-4, below GGUF Q8_0, and it collapses to 1.6e-13 at float64. Section 6 honestly shows the scheme is not epsilon-secure if the canonical hash is public, and only becomes perfectly secure under a keyed hash.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A global permutation of the hidden axis, propagated consistently through attention, gated MLPs, RMSNorm and input/output embeddings, leaves an L-layer transformer's function exactly unchanged while touching every parameter tensor.
- Permutation orderings can carry floor(log2 n!) bits, giving 714 KB of payload capacity in Mistral-7B and Llama-3-8B and 2.78 MB in GPT-OSS-20B, with decoding that needs no payload-specific script because the payload length is encoded in the leading bits.
- Random derangements across embedding, expert, KV-head and MLP inner dimensions neutralize 100% of parameters, versus 60% for NeuPerm, raising neutralization success from 0% to 100% on embeddings and 77% to 100% on attention queries for published payload families.
- Finite-precision arithmetic is the only source of behavioral divergence, which at bf16 is smaller than accepted near-lossless quantization and vanishes at f32 and f64.
- The stegosystem is not epsilon-secure under a public canonical hash because of a support gap, and becomes perfectly secure in Cachin's framework only when a keyed hash is used.
Strengths
The invariance is proved, not asserted, and extended through every parameter tensor
Theorem 1 composes Propositions 2-4 (MHA equivariance, gated-MLP equivariance, RMSNorm equivariance) with matched permutations of the input and output embeddings, and Appendix D carries it to GPT-OSS with Lemma 2 (expert order), Lemma 3 (KV groups and within-group heads) and Theorem 4, giving an explicit coverage list per tensor.
Neutralization improvement is demonstrated against three published payload families in Table 3
Against NeuPerm on TinyLlama-1.1B, embeddings go from 0% to 100% neutralization for MaleficNet, partial LSB, full LSB and sign-mapping, and self-attention from 77% to 100%, with the 77% failure traced analytically to TinyLlama's four KV heads leaving at least two fixed in 7 of 24 permutations.
The numerical-error hypothesis is tested directly rather than assumed
Honest security analysis that undercuts the attack's own stealth claim
Cost is benchmarked against quantizations practitioners already accept
Weaknesses
'Minimal reduction in model performance' is never measured as task performance
Neutralization evidence is confined to a single 1.1B model
No statistical treatment of the five-trial divergence experiments
The attack survives the threat model only by assumption
No artifact release and scarce implementation detail for the canonical ordering
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper takes one idea — that transformers are invariant to consistent reordering of their internal dimensions — and pushes it in both directions. Section 1 lists four contributions: global permutation symmetries that move 100% of LLM parameters as a defense; PermaNet as a new (theoretically) lossless payload-embedding attack; empirical neutralization of existing stegomalware and of PermaNet; and a performance-cost measurement claimed to be minimal. Scope is well chosen. Definitions are clean, the threat model in Section 2 is explicit about what is excluded (no trigger mechanisms, no end-to-end exploitation) and, importantly, about what is assumed (a bootstrap routine already running at load time, and an end user who neither fine-tunes nor quantizes). The headline numbers are concrete: neutralization coverage of 100% versus 60% for NeuPerm [12], and capacities from 173 KB for TinyLlama up to 16.68 MB for GPT-OSS-120B.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Can malware really be hidden inside the weights of a language model?
Yes, in the ordering of the weights rather than their values. This paper's PermaNet exploits the fact that a transformer gives identical outputs if its internal dimensions are consistently reordered, so the sequence of rows and columns acts as a code. Capacity is large for modern models: about 714 kilobytes in Mistral-7B or Llama-3-8B, 2.78 megabytes in GPT-OSS-20B, and a theoretical 16.68 megabytes in GPT-OSS-120B, which is enough for real payloads of the size used by earlier techniques such as MaleficNet.
How do you remove malicious data hidden this way?
Apply the same symmetry in reverse: randomly reorder the dimensions again. The defenders' version uses random derangements, where no row returns to its original position, applied across the embedding, expert, KV-head and MLP inner dimensions so that every parameter moves. In experiments this lifts removal success to 100% versus 77% or lower for the prior NeuPerm approach, at a cost smaller than the widely accepted 8-bit quantization.
Does hiding data in a model's weights make the model worse?
In exact arithmetic the reordering changes nothing at all. In real hardware finite precision introduces tiny rounding differences: on TinyLlama at bfloat16 the divergence between original and permuted outputs was 5.8e-4 in KL terms, below what 8-bit quantization causes, and rerunning at float64 collapsed it to 1.6e-13 with identical top-1 predictions. The paper does not, however, report standard task benchmarks such as perplexity, so the practical effect on capability is inferred rather than measured.
Can antivirus software detect this kind of hidden payload?
Not currently. Detecting it requires interpreting weights as tensors, recomputing a canonical ordering by hashing rows, ranking the permutation and converting that integer to bytes; no general-purpose scanner does this. The paper also shows an information-theoretic weakness: if the hashing scheme is public, some orderings become provably impossible for the encoder, letting a checker certify certain models as clean, though a secret keyed hash removes that leakage entirely.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Mathematical formulas are rendered unreliably in the extracted text, so the proofs in Appendix A and Theorems 1 and 4 were read with partially garbled notation; the arguments were nonetheless traceable.
- Figures 1 and the pseudocode layout of Algorithms 1-4 lost indentation and structure, though their content was recoverable.
- Table 1 columns for MLP capacity unintentionally run together for some rows; the total-bits and KB columns were used.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.