Biomedical Reference Generation Remains Unreliable across 26 Large Language Models
Maxim Topaz, Zhihong Zhang, Nir Roguin, Pallavi Gupta, Zichao Li, Laura-Maria Peltonen
cs.CL
Quick verdict
Rigorous 26-model audit: 55.4% of LLM-generated biomedical references are fabricated; verifiability ≠ accuracy, so verification before use remains essential.
Summary
Researchers at Columbia University and collaborators prompted 26 large language models, released between 2023 and 2026 by eight developers, to supply a missing reference for each of 69 biomedical passages spanning ten domains. Of 24,518 classified responses, 55.4% were fabricated — titles matching no paper in PubMed, Crossref, OpenAlex, or Google Scholar — while 30.9% were verifiable, 12.0% were partial matches, and 1.7% were declined. Fabrication ranged from 10.2% (Claude Opus 4.8, which declined 52.1% of prompts) to 98.4% (Ministral 3B). Among verifiable references, journal and year were usually correct (97.1% and 95.8%), but author lists were correct in only 51.7% of cases, typically retaining the true first author while inventing coauthors; 14.9% of all responses were correct in every evaluated field, rising to 31.8% for the five 2026-era models. The authors conclude that even a citation naming a real paper cannot be used unchecked. Limitations include descriptive-only model comparisons under unequal sampling (382–1,378 responses per model), confounding of model generation with training-data recency, absence of retrieval-enabled systems, reliance on four bibliographic indexes as ground truth, and code and data not yet deposited.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 4
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Across 26 models (24,518 responses), 55.4% of generated biomedical references were fabricated and no model eliminated fabrication.
- Verifiability is not bibliographic accuracy: among 7,581 verifiable references, journal (97.1%) and year (95.8%) were nearly always right but author lists only 51.7%.
- Author-attribution errors are systematic — models typically retain the correct first author and invent the remaining coauthors.
- Five 2026-era models improved to 35.3% fabricated and 31.8% fully correct, still below the reliability needed for unverified use.
- Declination behavior (Claude Opus 4.8 declining 52.1% of prompts) materially changes how a model's fabrication rate should be read.
Strengths
Scale and coverage
24,518 classified responses across 26 models from 8 developers spanning March 2023–May 2026, over 69 passages in ten biomedical domains (Methods; Table S1 with per-model rows summing to 100%).
Deterministic, LLM-free verification pipeline
Titles matched against PubMed, Crossref, OpenAlex, Google Scholar via difflib.SequenceMatcher at ratio ≥0.90 with manual review of every candidate; no language model in the classification loop (Methods), avoiding the circularity common in hallucination audits.
Separates verifiability from bibliographic accuracy
Control structure evidences de novo fabrication
Transparent sensitivity analyses
Weaknesses
Descriptive-only comparisons under unequal sampling
Manual-review reliability unreported
Generation confounded with training-data recency
Ground truth limited to four indexes
Artifacts not yet available
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This empirical audit (Columbia University School of Nursing with Tel Aviv Sourasky Medical Center and the University of Eastern Finland; arXiv 2609.14988) measures how often large language models fabricate biomedical references. The design is clean: 69 passages from published papers in ten domains, each with one reference replaced by a placeholder — 50 seeded with references previously identified as fabricated in the published record, 19 real references as controls. 26 models from eight developers (March 2023 – May 2026) were prompted via API to supply the complete missing reference. Central claims: (1) 55.4% of 24,518 classified responses were fabricated; (2) verifiability is not accuracy — among 7,581 verifiable references, author lists were correct in only 51.7% although journal (97.1%) and year (95.8%) were nearly always right; (3) errors are systematic, with models retaining a true first author and inventing coauthors; (4) five 2026-era models improved to 35.3% fabricated and 31.8% fully correct but remain unusable unverified.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What percentage of references generated by AI language models are fake?
In this study, 26 large language models were asked to complete missing biomedical citations, and 55.4% of 24,518 responses were fabricated — the generated title matched no paper in PubMed, Crossref, OpenAlex or Google Scholar. For the five models released in 2026 the fabrication rate was lower, 35.3%, but no model eliminated fabrication entirely.
Which AI model is most reliable for generating references?
On this benchmark, Claude Opus 4.6 was bibliographically correct in every evaluated field in 54.6% of its responses and GPT-5.5 in 48.1%. Claude Opus 4.8 showed the lowest fabrication rate (10.2%) but declined to answer 52.1% of prompts. Because every model erred frequently, the authors advise verifying any model-assisted reference before use.
If the AI cites a real paper with a working DOI, is the citation trustworthy?
Not necessarily. Among references that named a real indexed paper with a resolving identifier, the journal was correct 97.1% and the year 95.8% of the time, but the author list was correct in only 51.7% of cases — models often keep the true first author and invent the remaining coauthors.
Is the study's data or code publicly available?
Not yet at the time of writing: the prompts, model list, scoring script and aggregated per-model results are promised for deposit in Zenodo at proof stage, and the source passages are withheld because releasing them could identify published papers containing fabricated references.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1 was not analyzable as an image; per-model values were taken from the prose and Table S1 text.
- The abstract in the provided text is truncated mid-sentence (Conclusions. Fabric…); conclusions were reconstructed from the full text's discussion and closing paragraphs.
- Table S1 was extracted as run-on plain text; per-model percentages were parsed from it, so small transcription ambiguity is possible.
- Several evaluated models (e.g., GPT-5.5, Claude Opus 4.8) are 2026-era releases that could not be externally verified in this offline review.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.