Linguistic Triggers of Gender and Racial Bias in Open-Weight LLMs Applied to Recruitment
Kosuke Kitahara, Nobuhiro Yamaguchi
cs.CL · cs.AI · cs.CY
Quick verdict
Job-posting wording triggers gender/racial bias in six small open-weight LLMs; strong design and full stimuli, but pooled tests ignore non-independence.
Summary
The paper audits how job-posting language, rather than candidate attributes alone, triggers demographic bias in six open-weight LLMs (Llama 3.2, Mistral, Gemma 3, Qwen 3, Phi 3, DeepSeek-R1 7B) run via Ollama at temperature 0 with structured JSON output. Across four experiments — a recruiter role rating recommendation strength and a job-seeker persona rating interest (1–10) — the authors vary agentic vs. communal vocabulary (40 synthetic postings) and inclusive vs. coded-exclusion language (20 matched pairs). Agentic wording depressed female candidates' recruiter scores (Δ=+0.82, r_rb=0.309, p_Bonf=7.2×10⁻⁵; fixed-effects r_rb=0.448); the description body, not the title, drives it. Coded-exclusion wording cut non-White recruiter scores by 1.57–2.20 points (r_rb=0.646–0.758) versus 1.00 for White candidates, and selectively deterred non-White job-seeker interest; a no-label ablation attributes the differential to the explicit demographic label. Under inclusive language, White candidates scored lowest ('prosocial overcorrection'). WEAT on two separate embedding models corroborates associations (d=1.01–1.45). The paper proposes a four-stage pre-deployment audit protocol tied to EU AI Act Annex III and EEOC four-fifths analysis. Main limitations: pooled tests treat non-independent model×stimulus observations as independent; gender stimuli confound vocabulary theme with occupation content; candidate profiles are empty; no confidence intervals.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Agentic job-posting language significantly depresses recruiter recommendation scores for female candidates (Δ=+0.82, r_rb=0.309, p_Bonf=7.2×10⁻⁵; fixed-effects r_rb=0.448), with the description body rather than the job title as the primary driver.
- Coded-exclusion language suppresses non-White recruiter scores at large effect sizes (r_rb=0.646–0.758) and selectively deters non-White job-seeker personas' stated interest, operationalizing a chilling effect at scale.
- A no-label ablation (H1) shows the explicit demographic persona label, not posting language alone, drives the differential racial penalty (language-only baseline Δ=−0.73 versus label increments up to −0.87 for Black personas).
- Under inclusive language the models recommend White candidates least ('prosocial overcorrection'), a reverse disparity the authors treat as an adverse-impact-relevant audit finding.
- The findings support a four-stage pre-deployment audit protocol — vocabulary scoring, persona-conditioned probing, four-fifths adverse-impact flagging, and Annex III documentation — for high-risk recruitment AI under EU and U.S. frameworks.
Strengths
Genuine design novelty: posting language as the manipulated variable
Unlike prior LLM hiring audits (An et al. 2025; Wilson and Caliskan 2024) that hold postings fixed and vary candidate attributes, Sections 2.2 and 3.2 systematically manipulate the agentic–communal and inclusive–coded-exclusion registers across both gender and race within one framework, with all 80 stimuli reproduced verbatim in Appendix B.
Dual-perspective asymmetry is itself an informative result
The recruiter-side female penalty (r_rb=0.309) largely vanishes under job-seeker personas (r_rb=0.076, n.s.; Table 2), and coded-exclusion language selectively suppresses minority personas' stated interest (Δ=−1.60 for Black personas; Table 3) — a chilling-effect measurement that callback audits cannot observe, as argued in Section 5.3.
Layered robustness work targeting specific confounds
Unusually complete reproducibility materials
Candid limitations and positionality
Weaknesses
Pooled statistics ignore non-independence
Theme confounded with occupation and content in gender stimuli
Degenerate candidate profiles make the label the dominant signal
No uncertainty quantification
WEAT evidence is only indirectly connected to the behavioral findings
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What did this paper find about LLM bias in hiring?
It reports that agentic ('competitive, decisive') job-posting wording lowered recommendation scores for female candidates in six open-weight LLMs, while 'cultural fit'-style coded-exclusion wording lowered scores most for Black, Hispanic, and Asian candidates, and also reduced those groups' stated interest when the model played a job-seeker persona — a simulated 'chilling effect'.
Which AI models were tested in this study?
Six open-weight models of roughly 3B–8B parameters, run locally via Ollama: Llama 3.2 3B, Mistral 7B v0.3, Gemma 3 4B, Qwen 3 8B, Phi 3 Mini (3.8B), and DeepSeek-R1 7B (distill). Closed frontier models such as GPT-4 were not tested.
What is 'prosocial overcorrection' in this paper?
The authors' term for their finding that under inclusive, equity-oriented job ads the models gave White candidates the lowest recommendation scores (7.16 vs. 8.26–8.45 for the non-White groups on a 1–10 scale) — a reverse disparity they treat as an audit finding relevant to adverse-impact analysis regardless of which group it favors.
Do these results apply to GPT-4, Claude, or Gemini?
Not directly. The study covers only small open-weight models (3B–8B parameters); the limitations section explicitly warns that findings may not transfer to closed frontier models, larger open-weight variants, or different decoding settings.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1 and 2 are available only as captions and inline descriptions; the plotted distributions cannot be independently verified from the extracted text.
- Per-model result details appear only partially in the main text (three models named for the gender effect; Mistral summarized for the coded-exclusion pattern), so per-model claims could not be fully checked.
- This is the arXiv v1 extended camera-ready text; no code-repository statement was found — codeAvailable=false reflects the absence of a stated release, not a stated refusal, while stimuli and prompts are fully reproduced in Appendices A–B.
- The review assumes the provided extraction is complete; any omitted supplementary material (e.g., per-item statistics) could change reproducibility and statistics assessments.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.