INDRA: A New AI Tool for Exploring Tobacco, Fossil Fuel, and Chemical Industry Archives
Daniel Akselrad, Robert N. Proctor
cs.DL · cs.CL · cs.CY
Quick verdict
INDRA walls Claude inside litigation archives with provenance tags and a quote watchdog; strong case studies, but evaluation is small and baselines missing.
Summary
The paper presents INDRA (Industry Documents Research and Analysis), a platform built on Anthropic's Claude API that lets researchers interrogate litigation-disclosed industry archives — UCSF's Industry Documents Library, ToxicDocs, SRITA, and others — through a closed evidentiary sandbox. Three safeguards are described: (1) the sandbox confines the model to user-selected documents with a 500,000-token ceiling and blocks external retrieval; (2) sentence-level provenance tags ([DK] domain knowledge, [INT] interpretation, [PS] perspective shift, [T] transparency) distinguish archival evidence from parametric inference, verified by a deterministic watchdog script rather than model self-report; (3) a thirty-instruction system prompt enforces citation discipline, verbatim extraction, and disclosure of negative findings. Trimodal search combines BGE-M3 semantic search over a 540-million-edge knowledge graph, Boolean search, and LLM-free KWIC concordance ('Ecology'). Three case studies — the Tobacco Institute's 'College of Tobacco Knowledge,' Exxon's employee-magazine climate denial, and ICI/Syngenta's paraquat-Parkinson's strategy — demonstrate novel findings: zero occurrences of 'anthropogenic' across the archive's fossil fuel magazines, zero 'climate science' in a ~1,800-magazine set, and undisclosed industry ties of cited scholars. Preliminary benchmarks report zero fabricated quotations out of 3,643, versus 5.1–9.2% misgrounding for off-the-shelf Sonnet 5. Stated limitations include token ceilings, non-determinism ('Heraclitus effect'), conversational path dependency, and gullibility toward adversarial archives; evaluation lacks comparisons with competing RAG tools, and full benchmarks reside in externally hosted appendices.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- INDRA's combined safeguards (closed sandbox, deterministic provenance tagging, system-level protocol with quotation watchdog) reduce fabrication to zero — 0 of 3,643 quotations — and misgrounding to 0.5–0.6%, versus 5.1–9.2% misgrounding for off-the-shelf Claude Sonnet 5 on identical document sets.
- Federating siloed litigation archives (UCSF IDL, ToxicDocs, SRITA, etc.) into one LLM-accessible corpus enables cross-industry analysis not previously practical.
- Deterministic, LLM-free concordance analysis can establish corpus-level absences (zero 'anthropogenic' and 'extreme weather' in the archive's fossil fuel magazines; zero 'climate science' in ~1,800 magazines) that function as historical evidence.
- The Transparency tool can surface undisclosed industry conflicts of interest by combining archival documents with OpenAlex, PubMed Central, and Scopus metadata, anchored deterministically to disclosing text.
- LLM-based archival analysis has intrinsic limitations — non-determinism, prompt-order path dependency, and gullibility toward adversarial documents — that make outputs 'designed to be checked rather than trusted.'
Strengths
Deterministic provenance contract goes beyond citation-linking
Section 2.2–2.3: sentence-level [DK]/[INT]/[PS]/[T] tags plus a non-LLM 'watchdog' that fuzzy-verifies every quotation against source OCR before delivery (note 6), explicitly contrasted with Self-RAG's self-reporting (note 28) — a real design advance over standard RAG.
Concrete, falsifiable preliminary benchmarks
Note 7 / Appendix A.1: 0 fabrications in 3,643 quotations with safeguards; bare Sonnet 5 misgrounds real quotes 5.1% (light) and 9.2% (heavy loads); 65% unverifiable quotations with no documents; 96%→61% accuracy as context fills to 1M tokens (note 22).
Case studies are documented and checkable
Unusually candid limitations section
Weaknesses
No head-to-head evaluation against competing systems
Benchmark methodology externalized
Watchdog error profile unreported
No user evaluation
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
INDRA (arXiv:2609.11261, cs.DL) is a system paper by Akselrad and Proctor (Stanford) describing an LLM platform for interrogating litigation-disclosed industry archives: UCSF's Industry Documents Library (~150 million pages), ToxicDocs, SRITA, Trinkets & Trash, FBarchive, OxCCAL, rjrtdocs.com, plus proprietary collections. Three central claims: (1) off-the-shelf chatbots lack access to these archives and hallucinate when asked about them — 65% of bare Sonnet 5 responses with no documents contained unverifiable quotations, none with resolvable document IDs (note 7); (2) three safeguards — a closed evidentiary sandbox with a 500,000-token document cap, sentence-level provenance tagging ([DK]/[INT]/[PS]/[T]) verified deterministically, and a thirty-instruction system prompt backed by a non-LLM quotation watchdog — jointly suppress fabrication (0/3,643 quotations) versus 5.1%/9.2% misgrounding for bare Sonnet 5 under light/heavy loads; (3) three case studies (Tobacco Institute's 'College of Tobacco Knowledge'; Exxon's employee-magazine climate denial; ICI/Syngenta's paraquat-Parkinson's strategy) demonstrate novel, verifiable findings. The writing is literary and argumentative rather than conventionally empirical; some claims ('near-instantly one can reconstruct chronologies, typologies, meaning and intent') outrun what is demonstrated.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is INDRA?
INDRA (Industry Documents Research and Analysis) is a free research platform hosted by Stanford (indra.stanford.edu) that lets users search and analyze litigation-disclosed industry archives — UCSF's Industry Documents Library, ToxicDocs, SRITA and others — using Anthropic's Claude, confined to user-selected documents, with every claim tagged as archival evidence versus model inference, and every quotation checked against source OCR by a deterministic script.
How does INDRA prevent hallucinations?
Three mechanisms: a closed sandbox blocking external retrieval and capping context at 500,000 document tokens; a 30-instruction system prompt requiring verbatim extraction and document-ID citations; and a non-LLM 'watchdog' script that fuzzy-checks each quotation against the original OCR and removes fabrications before output. The paper reports zero fabricated quotations in 3,643 tested quotations, versus 5.1–9.2% misgrounding for bare Claude Sonnet 5.
What are the Heraclitus effect, the steppingstone dilemma, and the gullibility problem?
The paper's names for three LLM limitations: (1) the Heraclitus effect — the same query rarely yields identical answers because generation is non-deterministic; (2) the steppingstone dilemma — early questions in a session steer later answers via path dependency and sycophancy; (3) the gullibility (mafia) problem — the model may take deceptive industry documents at face value rather than reading them as strategic rhetoric written for future litigation.
Is INDRA open source, and can its results be reproduced?
No code release is described in the paper. The platform is freely usable at indra.stanford.edu and benchmark appendices are hosted at indra.stanford.edu/methods/appendices/, but the system prompt text, watchdog implementation, and embedding/knowledge-graph pipeline are not published, so the reported benchmark numbers are not independently reproducible; individual case-study quotations can be spot-checked via the linked public archive documents.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–6 are present only as captions in the extracted text; their visual content could not be assessed.
- Quantitative benchmark details (Appendices A.1–A.4) are cited in the paper but hosted externally, so their contents could not be verified from the provided text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.