ALIBI: Adversarial Legitimacy Injection in Binary Input against LLM Malware Analyzers
Hyeongjun Choi, Wonyoung Jung, Haehoon Seo, Sungyup Nam
cs.CR · cs.LG
Quick verdict
Claims a semantic cover-story flips 30/35 PE samples to benign on Gemini 2.5 Pro; novel, well-measured attack study, limited by small samples and no code.
Summary
The paper introduces ALIBI, an evasion attack against LLM-based malware analyzers that injects a coherent but false narrative into a compiled binary rather than explicit prompt-injection text. The payload — a non-executed read-only section plus forged identity strings, per-API rationales in release-notes prose, and aligned metadata (version, timestamp, size padding) — reframes suspicious evidence as expected behavior of an endpoint-security product while leaving imports and executable behavior untouched. On a frozen set of 50 MalwareBazaar PE samples, 35 of which Gemini 2.5 Pro labeled malicious at baseline, the integrated payload flips 30 (85.7%) to benign on Gemini; GPT-5.5 Pro and Claude Opus 4.7 show zero strict flips but substantial soft downgrades (71.4% and 42.9%) plus statistically significant self-reported confidence erosion (Wilcoxon p=0.001–0.021). The attack transfers to ELF (16/40 flips on Gemini). A verification-guided system prompt halves strict flips (82.9%→42.9%, McNemar p=0.0001) but does not close the gap; the authors argue for provenance checks separating verified facts from attacker-controlled strings. Limitations: small frozen samples, single query per sample, closed models, an undefended-arm reproduction of 82.9% vs the earlier 85.7%, a defense evaluated only on Gemini, and no released code or payload.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A semantic cover-story payload added as a non-executed read-only section flips 30 of 35 baseline-malicious PE samples to benign on Gemini 2.5 Pro (85.7% Strict ASR), while token-level prompt injection flips none on the same samples.
- GPT-5.5 Pro and Claude Opus 4.7 show zero strict flips but substantial soft downgrades (71.4% and 42.9%) plus statistically significant self-reported confidence erosion on unchanged verdicts.
- The attack transfers to ELF with 16/40 flips on Gemini 2.5 Pro, and attack strength tracks the alignment between cover story and the sample's static fingerprint.
- A verification-guided system prompt halves benign verdicts (82.9%→42.9%, McNemar p=0.0001) but leaves 42.9% reaching benign, so evidence builders need provenance checks separating verified facts from attacker-controlled strings.
- The same binary modification also causes Windows Defender to miss 10 of 39 originally detected PE samples, via structural perturbation rather than semantic reframing.
Strengths
Novel attack class with a controlled empirical contrast
ALIBI reframes evidence semantically instead of issuing override instructions; on the same channel, model, and samples, a token-level injection variant flips 0/35 while ALIBI flips 30/35 on Gemini 2.5 Pro (§1, §2.2) — a clean mechanism isolation rarely seen in evasion papers.
Multi-dimensional, conservative success metrics
Beyond Strict/Soft ASR (§5.2), the paper reports mean tier shift and paired confidence erosion, capturing queue-prioritization harm (mal→sus downgrades) and subliminal effects (Wilcoxon p=0.001–0.021, Table 3) that label-only evaluations would miss.
Mechanism-level findings, not just numbers
Honest scoping and disclosure posture
Weaknesses
Run-to-run instability of the headline number
Cross-model results measured against Gemini's baseline
No uncertainty quantification on any proportion
Defense evaluated narrowly
Reproducibility is low by construction
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
ALIBI (78ResearchLab) addresses a timely question: can the attacker-controlled static evidence inside a compiled binary steer frontier LLM malware analyzers without explicit prompt-injection strings? The central claim is that a semantic cover story — a non-executed read-only section carrying a false endpoint-security-product identity, release-notes-style rationales for suspicious APIs, and cross-checked metadata (§4.2) — reframes visible malicious evidence as expected tool behavior. Three supporting claims: (1) the integrated payload flips 30 of 35 baseline-malicious PE samples to benign on Gemini 2.5 Pro (85.7% Strict ASR, Table 2) while a token-level injection variant flips none; (2) GPT-5.5 Pro and Claude Opus 4.7 resist label flips but show substantial severity downgrades (Soft ASR 71.4% / 42.9%) and significant self-reported confidence erosion; (3) a verification-guided prompt halves but does not close the attack (§6.3), motivating provenance-aware evidence builders. Scoping is responsible: the paper explicitly disclaims applicability to dynamic analysis, sandboxing, and human reverse engineering (§3).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the ALIBI attack on LLM malware analyzers?
ALIBI is an evasion technique that adds a small never-executed read-only section to a malicious binary containing a fake but coherent story claiming the file is a legitimate endpoint-security product, along with forged metadata. When an LLM-based analyzer reads the extracted strings and metadata, it reframes suspicious behavior as expected security-tool activity and may return a benign verdict — without any explicit prompt-injection instructions being present in the file.
Which models did ALIBI fool, and by how much?
In the paper, Gemini 2.5 Pro flipped 30 of 35 baseline-malicious Windows PE samples to benign (85.7%). GPT-5.5 Pro and Claude Opus 4.7 never issued a benign label but downgraded 71.4% and 42.9% of those samples to lower-severity verdicts, with measurably reduced confidence. On Linux ELF samples, Gemini flipped 16 of 40, while GPT and Claude stayed below 10% soft downgrades.
Does ALIBI also bypass traditional antivirus like Windows Defender?
Partly, but through a different mechanism. The added section, strings, and metadata perturb file-level features used by signature and heuristic matching, so 10 of 39 PE samples Windows Defender originally detected were no longer detected after modification. Defender does not interpret the cover story semantically — this is a structural side effect, not a semantic bypass.
How can defenders mitigate this kind of attack?
The authors show that a verification-guided system prompt — treating embedded self-claims as attacker-controlled assertions to be checked against structural evidence — halves benign flips but still leaves 42.9% reaching benign. Their recommended architecture is provenance checking in the evidence builder: verify Authenticode signatures, PDB paths, and vendor claims against known product properties, and expose verified/unverified/contradicted fields to the model instead of trusting strings.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures arrived as raw text dumps; exact verdict distributions in Figure 3 and plotted values were checked only against in-text numbers, not against rendered figures.
- The payload narrative text is intentionally abstracted in the paper, so the precise wording that produced these effects is not available.
- Review is based solely on the provided extracted text; the reference list and appendix were included but not independently verified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.