The Audit Decides the Verdict: Instrument Effects Rival Demographic Bias in LLM Decision Audits
Siddharth Vohra, Manikandan Ravikiran
cs.CL · cs.AI · cs.CY
Quick verdict
Pre-registered 40k-request audit finds no LLM demographic bias surviving correction in any format; ranking exclusion is conclusive for hiring only.
Summary
Whether an LLM looks demographically biased may depend on how the audit asks. Lukk (2026) reported that, in charitable-aid allocation, the same models favor minority claimants when rating appeals one at a time yet penalize some when ranking side by side. This paper tests whether that reversal generalizes to hiring, lending, and medical triage. Using AgentFairBench profiles rendered under 40 validated name pairs, the authors issued 40,726 requests to five models across rating, binary-decision, and tie-permitting four-candidate ranking formats, with the primary test fixed before collection. The reversal does not appear: none of 36 planned contrasts survives Benjamini-Hochberg correction, and the pre-registered format-by-group interaction is +0.01 [-0.15, +0.17]. The rating advantage keeps its sign at roughly half the published size (+0.047 points in hiring), and a precision extension bounds any hiring ranking penalty at 0.052 positions against the 0.067 published margin - conclusive for hiring only, since the lending and triage floors (0.132, 0.232) sit above it. Planted disparities and a directional replication on the original aid materials bound these nulls. Instrument effects dominate: models recognize transparent audits, tie all identical-content comparisons, and favor first-listed candidates (0.094 SD) versus a maximum demographic effect of 0.049 SD. Limitations include name-only cues, twelve profiles per domain, and a tie-permitting ranking design that departs from the forced-order audits under test.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- The rating-advantage/ranking-penalty reversal reported for charitable aid does not generalize to hiring, lending, or medical triage: none of 36 planned contrasts survives Benjamini-Hochberg correction.
- The rating advantage persists attenuated at roughly half the published size (+0.047 points in hiring vs the published +0.090), and any hiring ranking penalty is bounded below the published 0.067-position margin (0.052 positions, p=.013).
- Models almost always recognize transparent audits, tie 100% of identical-content comparisons regardless of whether the varying detail is race or a hobby, and the forced-choice probe produces large false positives on placebo bundles.
- First-listed position (0.094 SD) has a point estimate larger than the largest measured demographic effect (0.049 SD), so audit construction moves verdicts at least as much as demographics.
- On the original aid materials the same pipeline reproduces the direction of both published effects (intervals crossing zero), localizing the phenomenon to those materials.
Strengths
Exemplary pre-registration and analysis-family discipline
The primary endpoint (hiring Black Rate-minus-Rank interaction) was fixed before collection; five post-hoc extensions are tagged, kept out of the 36-contrast family, and their equivalence margins (0.090 rating points, 0.067 rank positions) were declared beforehand (Sections 2, Appendix C).
Nulls are bounded three independent ways, not left blind
Table 3 states per-outcome MDEs (worst 0.22 SD); planted triage disparities track injection (0.48 [0.33, 0.62] recovered of 0.50); and the original FairFund stimuli reproduce the direction of both published effects (-0.064 vs published -0.067 on Rank Asian; +0.193 vs +0.090 on Rate Black, Table 5).
The instrument audits itself
Scale with factorial design hygiene
Weaknesses
Ranking exclusion is conclusive for hiring only
Tie-permitting Rank departs from the audited claim
Sensitivity at the disputed effect sizes is weak
The pre-registered test itself is underpowered for the headline
External-validity ceilings: names only, few profiles
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper tests whether the format-dependent reversal reported by Lukk (2026) on FairFund-Bench - minority-favoring ratings under one-at-a-time judgment versus penalties in side-by-side ranking - generalizes beyond charitable aid to three regulated domains: hiring, lending, and medical triage. Central claims: (1) the reversal fails to generalize (none of 36 pre-planned contrasts survives Benjamini-Hochberg at q<0.05; the pre-registered hiring Black Rate-minus-Rank interaction is +0.01 [-0.15, +0.17]); (2) the rating advantage persists attenuated at roughly half the published size (+0.047 [+0.002, +0.092] points in hiring versus the published +0.090); (3) a precision extension bounds any hiring ranking penalty below the published 0.067-position margin (0.052 positions, p=.013), conclusive for hiring only since the lending and triage floors (0.132, 0.232) sit above it; (4) instrument effects - near-total recognition of transparent audits, content-identity ties, and a first-position preference of 0.094 SD versus a maximum demographic effect of 0.049 SD - rival or exceed demographic effects. The abstract is unusually precise about which exclusions are conclusive.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Does this paper show that LLMs are unbiased in hiring and lending?
No. It shows that, under its specific instrument (minimal prompts, name-only cues, temperature 0, tie-permitting ranking), no demographic contrast survived multiple-comparison correction, and effects smaller than its stated detection floors are not excluded. The authors explicitly warn against reading this as a fairness certificate for deployed systems.
What is the instrument effect the title refers to?
It is the influence of audit construction on verdicts: models almost always recognize transparent fairness tests, tie every candidate comparison where content is identical, and favor first-listed candidates (0.094 SD) with a point estimate larger than the biggest demographic effect measured (0.049 SD). The same demographics read differently depending on how the question is asked.
Does this refute the FairFund-Bench findings of Lukk (2026)?
Not directly. Rerunning the original aid materials through this pipeline reproduced the direction of both published effects, though with intervals crossing zero, so it is directional agreement, not a statistical replication. The paper localizes the reversal phenomenon to those materials and shows it does not appear in hiring, lending, or triage under its design (conclusively for rating and for hiring ranking).
Is the code and data available?
The authors state they release code, prompts, raw responses, and analysis scripts, with a consistency check asserting the request totals (40,726 requests, 88,576 per-stimulus observations); a project website is listed at siddvoh.com/biasinaudits.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-3 appear in the extraction only as captions; all numbers cited in this review come from the in-text tables and prose, not from reading plotted values.
- Appendices A-C are present through the name/profile robustness subsection, which ends on a complete sentence; no evidence of truncation at the far end of the provided text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.