LLMs as Oracles: Reliance on LLMs for Subjective Personal Questions
Myra Cheng, Lujain Ibrahim, Grace Liu, Michelle S. Lam, Vishakh Padmakumar, Nick Madibekov, Diyi Yang, Dan Jurafsky
cs.CY · cs.AI · cs.CL
Quick verdict
Claims LLM-as-oracle use is rising and unnoticed; multi-dataset, preregistered evidence is compelling but rests on LLM judges and self-selected donors.
Summary
The paper characterizes 'LLM-as-oracle' use — people treating LLMs as all-knowing authorities on subjective personal questions. The authors develop an eight-role typology pairing four oracle roles (Interpreter, Evaluator, Decision-maker, Ghost-writer) with four non-oracle counterparts, plus a human-validated LLM judge that labels prompts at scale. Applied to public data (68K prompts from WildChat and ThoughtTrace), the typology shows oracle use rising between 2023 and 2026, more prevalent among younger users, men, and frequent AI users. A privacy-preserving data donation tool analyzing 140K prompts from 52 US participants yields similar trends (oracle proportion r=0.80, p<0.001) and reveals widespread underestimation: only 12 of 52 participants correctly identified their most frequent role. A preregistered experiment (N=520) shows people send nearly twice as many oracle messages to a disclosed AI than to a perceived human (OR=2.15, 95% CI [1.65, 2.80]). Secondary analysis of a sycophancy study (95K prompts, 1,015 participants) links more sycophantic models to more oracle use; prompting and DPO interventions reduce oracularity but trade off response quality. Main limitations: reliance on LLM judges, self-selected WEIRD samples, a small donation cohort, correlational temporal trends sensitive to denominator choices, and no explicit multiple-comparison correction.
Review complete
Assessment at a glance
6
risk flags identified
● high 0
● medium 4
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- LLM-as-oracle use on subjective personal questions increased from 2023 to 2026 across public and donated usage data.
- Oracle use is more prevalent among younger users, men, and more frequent AI users.
- People systematically underestimate their oracle use: only 12 of 52 donors correctly identified their most frequent role, and oracle roles are disproportionately absent from self-reports.
- People send nearly twice as many oracle messages to a disclosed AI than to a perceived human (OR=2.15, 95% CI [1.65, 2.80]).
- Model behavior matters: sycophantic models elicit more oracle use, and prompting or DPO interventions reduce oracularity at some cost to response quality.
Strengths
Dual-role typology operationalizes relative offloading, not just topic prevalence
Each oracle role (Interpreter, Evaluator, Decision-maker, Ghost-writer) has a non-oracle counterpart (§3.2), developed from Tseng & Liang's nine roles plus bottom-up categorization of 100-prompt batches to saturation (§3.1), with conservative exclusions of self-disclosure to avoid overreporting.
Multi-source triangulation across 300K+ prompts
Convergent evidence from WildChat, ThoughtTrace (68K combined), 140,834 donated messages from 52 participants (§4.2), a preregistered N=520 experiment (App. D), and a 95K-prompt sycophancy study (§6.3) — each headline claim has at least two independent data streams.
Preregistered causal experiment with careful null checks
Unusually honest robustness reporting
Privacy-preserving data donation tool as a reusable method
Weaknesses
LLM-judge circularity with incompletely visible validation
Aggregate temporal trends vulnerable to cohort-composition confounds
Multiplicity and denominator sensitivity
Sample limitations vs abstract claims
Perceived-human condition uses an LLM roleplayer
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Review: LLMs as Oracles (arXiv 2609.14849)
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does 'LLM-as-oracle' mean in this paper?
It refers to people treating large language models as all-knowing authorities on subjective personal questions — asking the model to decide, judge, or interpret for them (e.g., 'Should I quit my job?') rather than to support their own reasoning (e.g., 'lay out my options'). The paper defines eight roles, four oracle roles (Interpreter, Evaluator, Decision-maker, Ghost-writer) paired with four non-oracle counterparts (Contextualizer, Deliberator, Strategist, Editor).
Did the study find that people increasingly use LLMs this way over time?
Yes, with caveats. Across public WildChat data (2023-2024) and 140K donated messages from 52 participants (2022-2026), oracle use rose significantly (e.g., overall oracle proportion r=0.80, p<0.001 in donated data). In WildChat, the Interpreter and Evaluator roles increased significantly, but under a stricter denominator including all prompts only the Interpreter increase remained significant (r=0.65, p=0.012), so the trend evidence is strongest in the donated data.
How did researchers analyze private ChatGPT or Claude conversations without reading them?
They built a privacy-preserving data donation tool: participants upload their chat export to a web tool, an LLM judge labels the messages locally in the analysis, and participants download a JSON file containing only the labels (no conversation text) to share with researchers. 52 US participants donated 140,834 messages this way, and the tool also showed each participant a personalized report of their usage patterns.
Can LLMs be changed to reduce this over-reliance behavior?
Partly, per the paper's experiments: telling the model the user seeks self-discovery (via prompting, whether hard-coded or adaptively selected) significantly reduces oracular responses, and DPO fine-tuning achieves comparable reductions on open models. However, there is a measured tradeoff: models that reduce oracularity most score lower on a reward model (ArmoRM), and crowdworkers preferred the original oracular answers for immediate helpfulness while preferring the less oracular answers when asked which would help them develop their own interpretation.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Extracted text omits several sections (parts of §4.1 methods, the §6 main-text narrative, and parts of Appendix A on role-judge validation); judgments rely on visible sections and figure captions.
- Role-judge human agreement statistics were not present in the visible extraction, so judge reliability could not be assessed directly.
- Figures are only available as caption text; exact CI bands, box plots, and per-window sample sizes could not be verified visually.
- Review reflects the arXiv v1 (Sep 2026) preprint; peer-review status unknown.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.