TRIS: A Tri-Layer Retrieval Integrity Sieve Against Knowledge Poisoning
Muhaimin Bin Munir, Akib Jawad Ononto, Nazia Shehnaz Joynab, Bhavani Thuraisingham, Latifur Khan
cs.CL · cs.CR · cs.IR
Quick verdict
TRIS cuts RAG poisoning ASR 67/87/64%→3/14/4% via three filters; commendably honest, but comparative wins rest on n=30 runs and adaptive levels 2-3 untested.
Summary
The paper introduces TRIS, a middleware defense against retrieval-stage knowledge poisoning in RAG, where a handful of crafted Trigger-Payload passages (following PoisonedRAG) dominate dense retrieval and steer generation toward attacker-chosen answers. TRIS filters the retrieved top-k set through three layers: cross-embedding-space clustering using an independent judge model (Sentence-BERT MiniLM, majority-keep), structural detection of trigger-payload artifacts via prefix Jaccard/n-gram thresholds, and an optional LLM consistency check that fails open. On 100-query subsets of Natural Questions, HotpotQA, and MS-MARCO with Contriever retrieval (k=50) and GPT-3.5-turbo, black-box attack success rate falls from 67/87/64% to 3/14/4%, white-box HotFlip ASR drops from ~74% to 27.8% with Layer 3 enabled, and poisoned-document MRR falls from 1.000 to 0.000. Against fairly-tuned RobustRAG and TrustRAG the system is competitive rather than dominant; the MS-MARCO advantage (4% vs 20% ASR) was measured with TrustRAG's consistency check disabled. A Level-1 adaptive evaluation shows paraphrased triggers evade the structural layer, while enabling Layer 3 halves ASR (32% to 15% on NQ) and raises clean accuracy 18 points at 16-19s/query latency. Key limitations: judge-aware and full white-box adversaries remain untested, baseline comparisons rest on n=30 single runs, and the authors' own ablation shows Layer 1's independent geometry does not drive the measured gains.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A middleware applying clustering, structural, and LLM-consistency filters reduces PoisonedRAG black-box ASR from 67/87/64% to 3/14/4% on NQ/HotpotQA/MS-MARCO (n=100, Contriever k=50, GPT-3.5-turbo).
- White-box HotFlip attacks require the geometric and consistency layers together, dropping from ~74% to 27.8% ASR with Layer 3 enabled on NQ.
- The sieve drives poisoned-document MRR from 1.000 to 0.000 while restoring clean Recall@5 and MRR, meaning it reorders rather than discards the candidate pool.
- Against architecture-aware (Level 1) adversaries, paraphrased triggers evade the structural layer but enabling LLM consistency halves ASR (32.0% to 15.0% on NQ) while raising clean accuracy 18 points.
- Against fairly-tuned TrustRAG and RobustRAG, TRIS is competitive rather than dominant, differentiated by worst-case profile and roughly 36x lower verification cost in the common black-box regime.
Strengths
Uncommon experimental honesty
Appendix B.3 reports that Layer 1's advertised independent geometry leaves ASR statistically unchanged (MiniLM 32.0% vs Contriever 33.0% on NQ, p=0.88); Section 6.2 withdraws any adaptive-ASR advantage over fairly-configured TrustRAG; Limitations discloses partial-iteration white-box numbers (9 of 10 iterations) and the unreconciled Table 1 vs Table 2 RobustRAG discrepancy.
Actionable layer ablation
Table 4 shows Layer 2 alone cuts black-box ASR to 4.0% at 8 ms/query, white-box HotFlip needs L1+L3 (27.8%), and the full system is worse than L1+L2 on black-box (9.0% vs 3.0%) due to Layer 3 over-filtering — directly justifying the recommended L1+L2 default with adaptive Layer 3.
Explicit adaptivity taxonomy with scoped claims
Verified fail-open semantics
Fair-baseline methodology
Weaknesses
Headline comparative advantage measured against a weakened baseline
Comparative claims rest on underpowered single runs
Core conjunction thesis tested only at adaptivity Level 1
Small corpus and thin statistical treatment
Generator-verifier coupling under-crosschecked
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
TRIS (arXiv 2609.00470) proposes a middleware defense against retrieval-stage knowledge poisoning of RAG systems, instantiating the PoisonedRAG threat model (Zou et al., 2025) with Split-and-Merge Trigger-Payload documents: five poisons per query, under 1% corpus ratio. The central thesis is that a successful poison must simultaneously satisfy embedding similarity, internal trigger-payload structure, and generative plausibility, so a defense should test all three constraints. Claims are scoped explicitly via a four-level adaptivity taxonomy (Section 3.2): headline results are Level 0 (static) attacks, Level 1 (architecture-aware) is evaluated empirically in Section 6.5, and Levels 2-3 are acknowledged as conceptual. This scoping discipline raises the baseline trust of the whole evaluation.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the TRIS defense for RAG poisoning?
TRIS (Tri-Layer Retrieval Integrity Sieve) is a middleware that filters documents retrieved by a RAG system before generation. It applies three checks: clustering the top-k passages in an independent embedding space (Sentence-BERT MiniLM) and keeping the majority cluster; a structural filter flagging documents whose first 20 tokens overlap heavily with the query (Jaccard/n-gram threshold 0.8), the signature of PoisonedRAG trigger-payload documents; and an optional LLM consistency check that drops passages contradicting the model's own parametric answer, failing open when unsure.
How effective is TRIS against PoisonedRAG attacks?
On 100-query subsets of Natural Questions, HotpotQA and MS-MARCO with Contriever retrieval (k=50) and GPT-3.5-turbo, black-box attack success rate drops from 67%, 87% and 64% under vanilla RAG to 3%, 14% and 4% with TRIS, while clean accuracy recovers to 58-76%. White-box HotFlip attacks are mitigated from about 74% to 27.8% when the LLM consistency layer is enabled, and poisoned documents' mean reciprocal rank falls from 1.000 to 0.000.
Does TRIS work against attackers who know about the defense?
Partially. Against architecture-aware attackers who paraphrase triggers, the structural layer is bypassed almost for free (about 4.97 of 5 poisons still reach the top-50), but enabling the LLM consistency layer halves ASR from 32% to 15% on Natural Questions while raising clean accuracy 18 points. Judge-aware (Level 2) and full white-box (Level 3) adversaries are discussed but not implemented, so robustness against them remains an open question the authors acknowledge.
What does TRIS cost in latency?
The default configuration (clustering plus structural filter) adds about 12 ms per query. The LLM consistency layer dominates cost at roughly 2.9 s per query always-on, rising to 16-19 s under live retrieval; an adaptive mode that fires only when the first two layers disagree keeps it occasional (21% of HotpotQA queries). In the common black-box regime TRIS needs roughly 36x fewer generator calls than RobustRAG's per-document isolation decoding.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-2 are extracted only as scattered label/caption text; the visual diagram content could not be assessed from this extraction.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.