Do AI Coding Assistants Check Before They Install? A Pre-Registered Demand-Side Audit of Trust Signals in the Research Software Supply Chain
cs.CR · cs.AI · cs.SE
Quick verdict
Pre-registered audit: AI assistants virtually never check trust signals before installing (0/2,114 verified); rigorous, but 9/1,920 positives limit inference.
Summary
This pre-registered measurement study asks whether AI coding assistants read supply-chain trust signals — SBOMs, signed releases, build provenance attestations, and SECURITY.md channel declarations — before installing research software. From an 87-project corpus, six HPC and quantum projects were drawn by a seeded, registered procedure; the author built nine modified repository copies per project (per-signal, wrong-issuer, all-signals, inconsistent-metadata, and control) and ran 1,920 registered trials: three models (local qwen2.5-coder:14b, claude-sonnet-5, gemini-3.5-flash-lite) under gated and autonomous harnesses, plus a 216-trial frontier supplement (claude-fable-5, gpt-5.6, Kimi-K2.6). Behavior was scored from container logs, not model self-report. Verification was near-absent: 9 of 1,920 registered trials (0.5%) opened any signal before installing, 0 of 384 controls did, and no trial among 2,114 ran a verification command. With zero control positives the registered regression was unfittable, so the registered permutation fallback applied (p = 0.50). Only attestation-present rose above the control interval (4/192, 2.1%), and all positives came from two projects and the Sonnet arm. A per-trial cost ledger, reconciled against provider billing, shows price did not predict verification ($4.11 per positive for Sonnet). Key limitations: networkless containers made most installs fail at dependency resolution; injected cryptographic material was study-issued; coded rationales and inter-coder agreement are deferred to a later version.
Review complete
Assessment at a glance
3
risk flags identified
● high 0
● medium 2
● low 1
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- AI coding assistants essentially never check supply-chain trust signals before installing research software: 9 of 1,920 registered trials (0.5%) opened any signal pre-install and 0 of 2,114 trials ran a verification command.
- Signal presence — valid, wrong-issuer, combined, or contradicted by the project's own metadata — had no measurable effect on behavior (registered fallback test, p = 0.50).
- Price did not buy verification: the only arm that verified repeatedly cost $0.103/trial (claude-sonnet-5), while the most capable and most expensive arm ($1.001/trial, claude-fable-5) verified nothing in 50 trials.
- Verification must be implemented in the harness that runs the assistant; neither the model nor a scripted approval step supplied it (gated 3 vs autonomous 6 positives in 960 trials each).
Strengths
Genuine pre-registration with anticipated failure modes
Protocol, seeded draw, screening log, analysis plan, and a frozen rate card were DOI-deposited (10.5281/zenodo.22062503) on 2026-08-22, eight days before the first trial; a registered fallback exact/permutation test existed precisely for the zero-control-positives case that materialized, and the single deviation (local model swap gpt-oss:20b → qwen2.5-coder:14b) is documented with diagnostic transcripts promised (§3.1).
Behavioral ground truth from container logs
Outcomes derive from file-access and command records with a strict taxonomy — opening the README excluded, a verification event requires an executed command with logged result (§3.5–3.6) — directly addressing unfaithful self-report (ref [22]); the 0/2,114 verification-event count is the paper's most defensible number.
Complete, reconciled cost accounting
Honest disclosure of design-bound results
Weaknesses
The design cannot test its registered hypothesis
Networkless containers undermine ecological validity
Extreme concentration of positives
Secondary outcomes deferred past this version
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This preprint (arXiv:2609.07754) reports what the author presents as the first demand-side measurement of whether AI coding assistants read published supply-chain trust signals — SBOMs, signed releases, SLSA-style build attestations, and SECURITY.md channel declarations — before installing research software. It is positioned as the controlled companion to the author's own supply-side audits (refs [10, 11]: 2 of 30 projects publish signed releases, none an SBOM; 83.9% of cross-checkable citation metadata disagrees). Three claims carry the paper: (1) verification is near-absent — 9 of 1,920 registered trials (0.5%) opened any signal before installing, 0 of 384 controls, and 0 of 2,114 trials ran a verification command; (2) signal presence had no measurable effect (registered fallback p = 0.50); (3) price did not buy verification ($0.103/trial for the verifying arm; $1.001/trial for the most capable arm that verified nothing), so verification must be built into the harness.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 3 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Do AI coding assistants verify software provenance or SBOMs before installing packages?
According to this pre-registered study of six research-software projects and 1,920 registered trials, essentially no: assistants opened a trust signal file before installing in only 9 trials (0.5%), and across all 2,114 completed trials not one assistant ran a command that actually verified a signature, attestation, or SBOM.
Which AI model was most likely to check supply-chain trust signals?
claude-sonnet-5 was the only model that repeatedly opened signal files (9 verification-action trials out of 360, 2.5%). A free local model (qwen2.5-coder:14b, 1,200 trials), the cheapest hosted model (gemini-3.5-flash-lite, 360 trials), and the most expensive frontier model (claude-fable-5, about $1.00 per trial, 50 trials) opened nothing — price did not predict verification.
Does this study mean SBOMs and signed releases are useless?
No. The paper shows that publishing these signals is necessary but not sufficient for research software, because the assistants measured did not read them — whether valid, forged, combined, or contradicted by project metadata. The authors conclude verification must be built into the agent harness rather than expected from the model or an approval prompt.
Is the study pre-registered and reproducible?
Yes. The protocol, seeded project draw, analysis plan, and pricing rate card were deposited on Zenodo (DOI 10.5281/zenodo.22062503) on 2026-08-22, eight days before the first trial on 2026-08-30. Tooling, raw transcripts, container logs, the per-trial cost ledger, and billing reconciliation are released; one deviation (replacing the planned local model before execution) is documented in a released deviations log.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The supplied metadata abstract cuts off mid-sentence, but the complete abstract appears at the start of the paper body; no content was lost.
- Figures are available only as captions in the extracted text; all numbers cited in this review come from the running text and tables, not from the plotted data.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.