In RAG We Trust? Measuring Robustness of Retrieval-Augmented Generation Under Document Poisoning
cs.CR · cs.AI · cs.CL
Quick verdict
Small but candid factorial study: RAG context poisoning drops Llama 3.1 8B accuracy from 77.9% to 43.5%; findings are suggestive, honestly limited.
Summary
The paper measures how poisoning of retrieved context degrades a RAG system built from a 4-bit quantized Llama 3.1 8B (llama.cpp, CPU), a MiniLM retriever, and a FAISS index over ~19,597 FEVER evidence sentences plus 10 added passages. Three rule-based corruption strategies (entity swap, number swap, negation) are crossed with four poisoning levels (0-3 of 3 retrieved passages) over 49 fact-checking queries (37 distinct), yielding 588 runs and 1,176 paired clean/poisoned generations at temperature 0.1. Accuracy falls from 77.9% clean to 43.5% with all passages poisoned; fooled rate rises 9.5% to 34.0% and abstention 21.1% to 51.7%. Entity swap flips the most previously correct answers (23.5% fooled) and provokes the most abstention; number swap stays flat while poisoned passages are a minority (10.2% fooled at 0-1) then jumps at 2-3 (26.5%/34.7%), supported by query-level bootstrap intervals and exact McNemar tests. The dominant failure mode is abstention rather than fabrication, and a lexical-overlap proxy of unsupported generation falls under attack. Main limitations: tiny redundant query set, unlocked decoding producing a 9.4-point residual at 0/3, brittle keyword labels, rule-based corruptions that confound inconsistency with substitution, a single small quantized model, and no code release statement.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Poisoning the retrieved context of a 4-bit Llama 3.1 8B RAG system degrades accuracy from 77.9% on clean context to 43.5% when all three retrieved passages are corrupted.
- Entity swap flips the largest share of previously correct answers (23.5% fooled rate averaged over levels) and provokes the most abstention (43.4%).
- Number-swap corruption stays flat while poisoned passages are a minority (10.2% fooled at 0-1 of 3) and jumps once they form a majority (26.5% at 2/3, 34.7% at 3/3).
- The model's dominant failure mode under attack is abstention (35.2% average) rather than fabrication, and the lexical-overlap proxy of unsupported generation falls rather than rises under poisoning.
Strengths
Candid self-audit of its own noise floor
Section III.B quantifies the 0/3 residual (9.5% fooled, accuracy 9.4 points below the clean arm, only 76.9% identical answer pairs) and explicitly downgrades light-poisoning contrasts to suggestive; the abstract carries the same hedging instead of hiding it.
Uncertainty treatment unusual for the sample size
Query-level bootstrap (10,000 resamples, fixed seed, percentile 95% CIs) plus exact McNemar tests on discordant pairs, with effect sizes primary; the number-swap jump is reported with its wide CI [2.0, 30.6] rather than as a clean discovery.
Internally consistent, complete factorial reporting
Paired design isolates the attack effect
Actionable failure-surface findings
Weaknesses
Tiny, redundant evaluation set
Decoding not locked across paired arms
Brittle correctness labels, no adjudication
Rule-based corruptions confound mechanisms
Single small quantized model; proxy of unknown validity
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks three practical questions about retrieval-augmented generation (RAG) under document poisoning: how far does accuracy fall, which corruption type bites hardest, and does the model fail by fabricating or by refusing. The testbed is a 4-bit Llama 3.1 8B (GGUF Q4_K_M via llama.cpp on an 8 GB CPU machine), a MiniLM retriever, and a FAISS flat index over 19,597 FEVER dev evidence sentences plus 10 hand-added passages (k=3). A full factorial crosses three corruption strategies (entity swap, number swap, negation) with four poisoning levels (0-3 of 3 passages) over 49 fact-checking queries (37 distinct), giving 588 runs and 1,176 paired clean/poisoned generations at temperature 0.1. Central claims: accuracy drops 77.9% to 43.5% at full poisoning; entity swap flips the most previously correct answers (23.5% fooled); number corruption jumps only when poisoned passages become a majority (10.2% to 26.5%/34.7%); abstention, not fabrication, dominates (35.2% vs 21.1% fooled averaged); and a lexical-overlap proxy of unsupported generation falls under attack.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is RAG document poisoning?
Editing or injecting the text that a retrieval-augmented generation system fetches, so the model reads corrupted evidence that still looks legitimate. This paper simulates it by rewriting retrieved passages after retrieval and before prompting — swapping entities, changing numbers, or inserting negations — without attacking the retriever itself.
How much does context poisoning hurt RAG accuracy in this study?
For a 4-bit Llama 3.1 8B answering FEVER-based fact-checking questions with three retrieved passages, accuracy fell from 77.9% on clean context to 63.9% with one passage poisoned and 43.5% with all three poisoned. The correct-answer flip (fooled) rate rose from 9.5% to 34.0% across the same sweep.
Does a poisoned RAG model hallucinate more?
Not in this study. The model's dominant reaction was to abstain (answering not-enough-information rose from 21.1% to 51.7% of runs), and a lexical-overlap proxy of unsupported generation actually fell under attack, from about 29% to 20%. The authors caution the proxy is coarse and may be suppressed mechanically by abstentions.
Which type of document corruption was most damaging?
Entity swaps (replacing, say, Paris with Madrid) flipped the most previously correct answers, a 23.5% fooled rate averaged over poisoning levels. Number swaps were harmless while only one of three passages was poisoned (10.2% fooled) but jumped once poisoned passages formed a majority (26.5% and 34.7% at two and three).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.