When Auditors Fabricate: Batch-Size Degradation and Confident Hallucination in LLM Detection of Planted Document Contamination
Karan Parekh, Sanjana Pendyala Ravinder, Sana Mhapsekar, Medina Maloku
cs.CL · cs.AI
Quick verdict
Finds LLM error detection collapses to 2.8% at large batch with fabricated findings; credible small study, but URL-retrieval confound leaves cause unproven.
Summary
The paper tests whether an LLM auditor reliably reports planted document defects as workload grows. The authors contaminate 150 real academic papers (supply chain and medical domains) with 450 logged defects in three classes — typographical corruption, semantic reversal, and absurd insertion — then score Google Gemini 3.0 Pro (free tier) on a 180-defect answer key across 60 documents in three regimes: single documents (2 docs), small batches (10 docs), and large batches (48 docs in six batches of eight). Recovery is 50% (3/6) for single documents, 60% (18/30) for small batches — a difference the sample cannot resolve — and 2.8% (4/144) for large batches. At scale the model does not abstain: it returns fluent, correctly formatted audits filled with invented defects ('telepathic squirrel', 'quantum-powered toaster') that appear nowhere in the corpus, formally indistinguishable from genuine findings. Absurd insertions were recovered most often (75% in completed evaluations) versus 50% for the more plausible typo and reversal types. Key limitations: one model, one scored run per regime, only 180 of 450 defects scored, and — acknowledged as the principal confound — documents were always supplied by URL, so batch size is entangled with retrieval load and no mechanism is established. Code, corpus, and detection logs are public.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 19 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Detection of planted contaminants collapses from 50–60% at single-document and small-batch scale to 2.8% (4/144) at large batch (Section 4.1).
- At large scale the failure mode is confident fabrication rather than abstention: the model invented genre-consistent contaminants (e.g., 'telepathic squirrel') absent from every document (Section 4.2).
- Fabricated audits are fluent, correctly structured, and formally indistinguishable from genuine findings, leaving downstream consumers no available failure signal (Section 4.2).
- Detection is type-asymmetric: absurd insertions were recovered at 75% in completed evaluations versus 50% each for semantic reversals and typos, so the most plausible corruptions are missed most (Section 4.3).
- Batch size is confounded with retrieval load because documents were always supplied by URL, so the mechanism of the collapse is not established by this study (Sections 5.2 and 6).
Strengths
Unusually candid statistical self-limiting
Wilson 95% CIs per regime (Section 4.1: Part I 18.8–81.2%, Part II 42.3–75.4%, Part III 1.1–6.9%); the authors declare Parts I vs II indistinguishable (z=0.45, p=0.65) and explicitly 'claim one effect rather than a graded curve', resisting a dose-response narrative their table invites.
Concrete, mechanically verifiable fabrication evidence
Section 4.2 names seven invented contaminants ('quantum-powered toaster', 'telepathic squirrel', 'disco-dancing warehouse', etc.), verifies none appear in any corpus document, and contrasts them with the 4/144 actual recoveries in the same documents.
Disclosure of a corpus weakness most authors would bury
The principal confound is identified by the authors themselves, with a replication recipe
Full artifact release
Weaknesses
Title asserts a mechanism the design cannot support
The 50% single-document baseline rests on n=2 documents / 6 contaminants
One model, one scored run per regime, free tier
Free-tier serving is an unexamined alternative mechanism
Only 180 of 450 contaminants scored, and precision is unmeasured
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks a deployment-shaped question: when an LLM auditor is handed more documents than it can genuinely process, does it abstain or fabricate? The authors build a contaminated corpus of 150 published papers (supply chain management; medical/pharmaceutical sciences), each carrying exactly three logged contaminants — typographical corruption, semantic reversal, and absurd out-of-context insertion — for 450 defects total, then score Google Gemini 3.0 Pro (free tier) on a 180-contaminant answer key over 60 documents at three scales. Central claims: (1) recovery collapses from 50% (3/6) and 60% (18/30) in Parts I/II to 2.8% (4/144) in Part III; (2) the large-batch failure is confident fabrication — invented, genre-consistent contaminants such as 'telepathic squirrel' — not abstention; (3) fabricated audits are formally indistinguishable from genuine ones; (4) detection is type-asymmetric (75% absurd vs 50% typo/reversal within completed evaluations, Section 4.3). The claims are unusually well hedged; the paper flags its own principal confound.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What happens when an LLM is asked to audit too many documents at once?
In this study, Google Gemini 3.0 Pro recovered only 2.8% of planted errors when auditing 48 documents in six batches of eight, versus 50–60% on single documents and small batches. Crucially, it did not report being overloaded: it returned fluent, well-formatted audit reports containing invented errors like 'telepathic squirrel' and 'quantum-powered toaster' that appear nowhere in the documents, indistinguishable in form from genuine findings.
Is 'planted contamination' in this paper the same as training-data contamination?
No. The authors use 'contamination' to mean deliberately injected defects inside documents an LLM is asked to audit — typos, reversed claims, and absurd inserted phrases. This is unrelated to the standard LLM-literature sense of benchmark or pretraining-data leakage, a distinction the paper makes explicitly in Section 2.
Did the study prove that large batch size causes the detection collapse?
No. In every regime the documents were supplied as URLs for the model to fetch, so batch size and retrieval load changed together. The authors name this as the study's principal confound (Section 5.2) and propose holding batch size fixed while comparing URL-referenced versus directly injected text as the first replication experiment.
Which model was tested and can the results be reproduced?
One model, Google Gemini 3.0 Pro on its free tier, with a single scored run per regime. The evaluation pipeline, prompts, scoring scripts, contaminated corpus, answer key, and full detection logs are published on GitHub, so the protocol is inexpensive to repeat — though the authors caution results reflect model access at evaluation time and may not transfer to other or later models.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The metadata abstract supplied with the paper cuts mid-sentence ('We conclude that LLM d'), but the complete abstract is present in the provided full text; the paper body is complete, ending with a full reference list — no truncation observed.
- Review is based on extracted text only; any figures the PDF may contain were not available for assessment.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.