InfraPatch: Cross-Task Targeted Grayscale Patch Attacks on Infrared-Adapted Vision-Language Models
Chengyin Hu, Dingyi Lu, Jiaju Han, Xiang Chen, Weiwen Shi, Jiahuan Long, Yiwei Wei, Jiujiang Guo
cs.CV · cs.AI
Quick verdict
Claims first cross-task targeted grayscale patch attacks on IR-VLMs (86–100% ASR); careful, honest scope, but one synthetic set, one target, white-box only.
Summary
The paper introduces InfraPatch, a white-box, per-instance framework of targeted digital grayscale patch attacks against infrared-adapted vision-language models (IR-VLMs). Within an approximately 5% area budget, a single square single-channel patch is optimized under task-adaptive objectives spanning classification, captioning, and binary visual question answering, with total-variation regularization and an early-exit proxy location search. Evaluation applies DiffV2IR to a fixed 30-category, 300-image COCO subset to synthesize infrared-style images, fixes the target semantic to 'giraffe', and uses clean-conditioned success criteria that exclude samples whose clean output already satisfies the target. Across ten LoRA-adapted variants, targeted attack success rates range from 86.00% (BLIP-2) to 100% (OpenCLIP and both LLaVA variants); proxy location search adds 6.67 and 10.33 percentage points on CLIP and BLIP-2 respectively, while LLaVA-1.5 is saturated; adapted structured infrared attacks (AdvIB, AdvIC, HCB) reach only 1.33-14.69% under the same protocol. Main limitations: a single synthetic distribution and a single target semantic, a purely white-box digital setting, baseline comparisons not matched for query or compute cost, and possible confounding from the authors' own LoRA adaptation.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A localized (~5% area) grayscale patch can inject one attacker-chosen target semantic ('giraffe') across classification, captioning, and binary VQA of infrared-adapted VLMs, with clean-conditioned targeted ASR of 86.00-100% across ten variants (Table 1).
- Early-exit proxy location search improves targeted ASR over optimized random placement by 6.67 pp on CLIP and 10.33 pp on BLIP-2, while LLaVA-1.5 stays saturated near 100% (Table 3).
- Adapted detector-era structured infrared attacks (AdvIB, AdvIC, HCB) reach only 1.33-14.69% targeted ASR under the identical protocol, indicating weak transfer of their geometries to targeted IR-VLM semantics (Table 2).
- Vulnerability and component utility are architecture- and task-dependent: BLIP-2 gains most from larger patches (76.67% to 89.67% from 3% to 7% area), CLIP rises gradually, and LLaVA shows a ceiling effect from ~3% to 7% (Table 4, Figure 4).
Strengths
Clean-conditioned protocol prevents inflated success rates
Sections 3.2-3.3 require clean correctness/non-target status for validity and count clean target hits in the denominator only; Table 1 reports Nvalid explicitly (classification 132-180, captioning ~300, VQA 286-300), making every ASR a conditional, auditable quantity rather than an end-to-end deployment probability (Section 6.7).
Unusually explicit scope discipline and negative results
Sections 2.1, 3.1, 6.7 and 7.1 repeatedly disclaim physical realizability, black-box transfer, EOT and universal patches; Section 6.3 discloses the placement control is not compute-matched (300 vs 340 updates), and Section 6.4/Table 4 honestly reports loss terms whose removal increases CLIP ASR (lambda_sup=0: 96.36%; lambda_orig=0: 97.58%).
Complete hyperparameter and baseline-budget disclosure
First unified cross-task comparison for IR-VLMs under one protocol
Weaknesses
Entire evidence base rests on one target semantic and one synthetic set
Baseline comparison conflates representation with optimizer and access regime
In-house adaptation confounds the IR-VLM vulnerability claim
No inferential statistics for controls and ablations
Success semantics partially deferred to code
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
InfraPatch (arXiv 2609.02233) claims the first systematic cross-task study of targeted adversarial patches for infrared-adapted vision-language models (IR-VLMs). Section 1 lists four contributions: a unified white-box framework optimizing a single square grayscale patch under a nominal 5% area budget; early-exit proxy location search coupled to task-adaptive objectives; evaluation over ten LoRA-adapted variants from the CLIP, BLIP, OpenFlamingo, and LLaVA families across classification, captioning, and binary VQA; and controlled comparisons with adapted AdvIB, AdvIC, and HCB. The central empirical claims are: clean-conditioned targeted ASR of 86.00-100% (Table 1); placement-search gains of +6.67/+10.33 pp on CLIP/BLIP-2 (Table 3); and 1.33-14.69% ASR for the adapted structured baselines (Table 2). Crucially, the scope is explicitly bounded to digital, white-box, per-instance attacks on one synthetic DiffV2IR-translated COCO set with one target semantic ('giraffe') - the paper never claims physical realizability, transfer, or universality, and Section 7.1 states each of these exclusions.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is InfraPatch?
InfraPatch is a white-box, per-instance adversarial attack method that optimizes a small (about 5% of image area) square grayscale patch to make infrared-adapted vision-language models produce an attacker-chosen semantic ('giraffe') - as a top-1 class, a captioned object, or a yes answer in visual question answering - evaluated entirely digitally on synthesized infrared-style images.
How successful is the InfraPatch attack?
Under the paper's clean-conditioned protocol (images whose clean output already names the target are excluded from the denominator), targeted attack success ranges from 86.00% (BLIP-2 captioning) to 100% (OpenCLIP classification and both LLaVA VQA variants) across ten model variants; adapted versions of earlier infrared attacks (AdvIB, AdvIC, HCB) reach only 1.33-14.69% under the same rules.
Does InfraPatch work in the physical world, e.g., printed on objects seen by thermal cameras?
No. The authors explicitly restrict all claims to a digital, white-box threat model: patches are optimized directly on preprocessed input images with full access to model gradients. No printing, sensor noise, viewpoint change, real thermal imagery, or thermal-material modeling is tested, and the single-channel constraint should not be read as a thermal material design.
What data and models were used in the evaluation?
Three hundred infrared-style images produced by applying the DiffV2IR visible-to-infrared diffusion model to a fixed 30-category COCO subset (10 images per category), evaluated on ten LoRA-adapted, frozen variants from the CLIP (4 classifiers), BLIP-2/InstructBLIP/OpenFlamingo (4 generation models), and LLaVA-1.5/1.6 (2 VQA models) families, with a single fixed target semantic ('giraffe').
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 2-4 panels are raster images not inspectable from the extracted text; assessment of qualitative examples relies solely on their captions.
- The exact COCO synonym map, baseline continuous fitness definitions, and LoRA training-set details are stated to live in the released code and are not reproduced in the paper text, so these specific rules were not verifiable here.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.