Understanding AI Provider Recommendations in Local Service Markets
cs.CY · cs.CL · cs.IR
Quick verdict
Registry-backed audit finds AI referrals largely fabricated without search and far more reliable with it; the within-model design is rigorous, though match-rate framing and weighting choices merit scrutiny.
Summary
The paper audits whether AI assistants' provider referrals are real and whether they track quality. Across five domains (primary-care doctors, hospitals, nursing homes, financial advisory firms, restaurants) and the 100 largest US metros, the authors issue 2,010 identical prompts to a proprietary model with and without native web search, plus an open-weight model (gpt-oss-120b) answering from memory, and match every recommended name against official registries (CMS clinician/facility files, SEC Form ADV) or Google Maps. Without search, doctor match rates are 4% (open-weight model, shown via a specialty-mix test to be name collisions) and 11% (proprietary); nursing homes 0.06% and 33%. With search, matches rise to 64-71% in these thin-coverage domains, while hospitals and firms move little. No-search recommendations over-select firms with SEC misconduct disclosures (3.6x the registry base rate, inverted to below base with search) and concentrate real referrals in the largest metros; search flattens both gradients. Citation analysis shows searched answers rest on .gov registries mainly for nursing homes (0.3% for doctors). Limitations: one proprietary model at one time point, conservative name+city matching that counts suburban or renamed providers as unmatched, different prompt grids between the open-weight and proprietary arms, and contested registry quality measures.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Without web search, AI provider recommendations in thin-web-coverage domains (doctors, nursing homes) are largely fabricated: 4% (open-weight) and 11% (proprietary) of recommended doctors match a clinician in the queried city, and the open-weight model's matches are name coincidences.
- Enabling search on the identical model raises match rates to 64-71% in those domains, shifts nursing-home recommendations strongly above roster quality, and inverts advisory-firm misconduct over-representation to below the registry base rate.
- Without search, real referrals concentrate in the largest metros, imposing a geographic penalty on smaller-market users; search largely removes this gradient.
- Restaurant referrals select on visibility (3-5x review-count premium) far more strongly than on quality (at most +0.11 stars).
- Searched answers' source mix varies sharply by domain (35.8% .gov citations for nursing homes vs 0.3% for doctors), and answers produced without retrieval carry no user-visible sign that recommendations were never verified.
Strengths
First audit linking AI referrals to official quality/misconduct registries
Section 4 matches recommendations to CMS clinician, hospital, and nursing-home files and SEC Form ADV Item 11, enabling quality-selection (Sec 5.3) and misconduct-composition (Sec 5.4) analyses that prior existence/demographics audits (Haupt et al.; Parikh et al.) could not perform.
Clean within-model search contrast
Arms B and C use the same model, prompts, decoding, and dates (Aug 21, 2026) with search toggled, so effects are attributable to retrieval configuration; the clustered B-to-C doctor gap is +53.1pp [47.5, 58.4] (Appendix G), and arm B was verified to carry zero retrieval citations.
The specialty-mix collision test is a genuine methodological contribution
Unusually thorough robustness program
Citation source-class analysis connects mechanism to consequence
Weaknesses
Conservative matching conflates 'unmatched' with 'fabricated'
Single proprietary model, single time window, and cross-grid arm comparison
Item 11 severity-agnostic aggregation and weighting-dependent headline
Restaurant arm carries a timing and serving-stack confound
The thin-versus-dense web-coverage mechanism is asserted, not measured
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Do AI chatbots make up doctor recommendations?
This audit found that without web search, only 4% (open-weight model) to 11% (proprietary model) of recommended doctors matched a real clinician in the queried city, and the open-weight model's few matches were coincidental name collisions, not genuine referrals. With web search enabled, 64% of recommended doctors matched a real clinician.
Does turning on web search fix AI-hallucinated referrals?
Largely yes for whether providers exist: match rates rose to 64-71% in the thin-coverage domains, recommendations shifted toward better-rated nursing homes and advisory firms with fewer SEC misconduct disclosures, and the disadvantage of users in smaller metros mostly disappeared. However, searched doctor answers relied almost entirely on commercial marketing and booking pages (0.3% government citations), so recommendations became real but not better than average.
Which sources do AI search answers cite when recommending doctors versus nursing homes?
In the audited system, 35.8% of nursing-home citations pointed to .gov hosts (88% of nursing-home responses cited at least one government source, mostly Medicare Care Compare), versus only 0.3% for doctors, whose answers cited health-system marketing and appointment-booking sites; restaurant answers cited no government sources at all.
Do AI assistants recommend the best restaurants or just the most popular ones?
Mostly the most visible ones: recommended restaurants had 2.9-5.3 times the median Google review count of typical operating restaurants but only 0.07-0.11 stars higher ratings, meaning the referral layer selects on visibility several times more strongly than on quality when both are separately measurable.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures were extracted as raw text; point estimates were taken from captions and body text, but interval widths and visual patterns in Figures 1-6 could not be independently verified.
- The supplied abstract in the metadata is truncated mid-sentence; the review relies on the complete abstract embedded in the full text.
- Appendix E per-cell tables and Appendix F unmatched-name rate tables are referenced but their contents are only partially present in the extraction.
- The proprietary model is identified only by its API name (gpt-5.6-terra), so replication of arms B/C depends on vendor availability and the exact API behavior at audit time.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.