LLM-as-a-Demographic: Whom Sociodemographic Prompting Helps, and Whom It Hurts
Daniela Occhipinti, Andrea Piergentili, Marco Guerini
cs.CL
Quick verdict
Finds demographic prompting moves LLM judges toward majority, away from minority groups; rigorous distributional study but only 3 English tasks.
Summary
The paper asks whether prompting an LLM judge with an annotator's demographic profile actually reproduces that group's judgments. The authors evaluate 23 open-weight models (five families, 4B–70B, including 11 matched base–instruct pairs) on three subjective tasks from the DEMO corpus — tweet intimacy, Reddit comment offensiveness, and workplace email politeness on 5-point scales — comparing model label distributions, read from answer-token log-probabilities and scored with an inverted Earth Mover's Distance, against real per-group annotator distributions across unconditioned, single-attribute, and intersectional profiles. Three results emerge. Unconditioned judges align best with White, college-educated annotators. Conditioning helps 10 of 11 base models but, for instruction-tuned judges, hides opposite per-group effects: gains for White/male profiles, losses for Black (−0.033 pooled), Asian, and female profiles, worst on offensiveness where Woman×Black reaches −0.087. Rating-shift analysis (e.g., Black annotators' real +0.052 deviation versus a +0.080 induced shift) shows prompts activate stereotypes rather than group perspectives, and the asymmetry points to instruction tuning as a plausible source. Limitations: three English tasks from one corpus, thin minority reference distributions (sometimes a single annotator per cell), and correlational rather than mechanistic evidence for the instruction-tuning attribution. Prompts and code are stated to be released.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Unconditioned LLM judges are not perspective-neutral: every evaluated model aligns most closely with White and college-educated annotators (§4.1).
- Demographic conditioning on average helps base models (10/11; +0.014 s, +0.032 mode accuracy) but has no average effect on instruct judges, concealing opposite per-group effects (§4.2).
- For instruct models, conditioning moves predictions toward majority profiles and away from minority ones (Black −0.033 pooled; offensiveness Woman×Black −0.087), concentrated on offensiveness (§4.3).
- Conditioning shifts reflect stereotype amplification rather than group reproduction — e.g., Black's real +0.052 deviation elicits a +0.080 shift, and Asian's negative politeness deviation (−0.027) elicits an upward shift (Table 4).
- Instruction tuning is the plausible source of the asymmetry, per 11 matched base–instruct pairs where base models improve for every group (§3.4, §4.2–4.3).
Strengths
Distributional evaluation instead of mean/majority labels
§3.2 scores predictions against each group's full 5-point label distribution via inverted, normalized Earth Mover's Distance (s ∈ [0,1]); Table 1's worked example (men and women share mean rating 0.375 but have different distribution shapes, yet Llama 70B scores s=0.91 vs 0.67) concretely motivates why aggregate-label scoring collapses.
Systematic confound controls
§F.1 density-matched references and §F.2 mode accuracy both target the sharpness artifact (instruct entropy 0.274 vs base 0.919); the base-model conditioning gain doubles under mode accuracy (+0.014 → +0.032, positive in 10/11 models) and the instruct race gaps survive both controls — the robustness section is unusually careful.
Matched base–instruct design across families and scales
Stereotype-shift quantification against real group deviations
Template robustness check
Weaknesses
Noisy gold standards exactly where harms are claimed
Heavily filtered demographic space
Correlational instruction-tuning attribution
Small effects, no multiplicity handling
External validity bounded to one corpus
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper evaluates sociodemographic prompting of LLM judges distributionally: does conditioning on an annotator's demographic profile move the model's label distribution toward that group's real judgments? Across 23 open-weight models (Gemma 3, Llama 3.1, Ministral 3, OLMo 3, Qwen 3; 4B–70B; 11 matched base–instruct pairs; §3.4) and three subjective tasks from DEMO — intimacy (MINT tweets), offensiveness (POPQUORN Reddit comments), politeness (POPQUORN emails), all 5-point ordinal (§3.1, Table 2: 7,210 texts, 43,402 judgments, 971 annotators) — it claims that (1) unconditioned judges default to White, college-educated perspectives (§4.1); (2) conditioning is asymmetric under instruction tuning, helping majority and hurting minority groups, worst on offensiveness (§4.3); and (3) instruction tuning is a plausible source (§4.2–4.3). Claims are explicitly scoped to open-weight English models, with honest non-transferability caveats (Limitations).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is sociodemographic prompting and what does this paper find?
Sociodemographic prompting tells an LLM judge to answer as if it were an annotator with a given demographic profile (gender, age, race, education). This paper tests 23 open-weight models on three subjective annotation tasks and finds that for instruction-tuned models the profile pushes predictions toward majority groups (White, male) and away from minority groups (Black, Asian, women) instead of toward the named group's real judgments — the opposite of the method's purpose.
What data and models does the study use?
The DEMO dataset combining MINT tweet-intimacy ratings and POPQUORN Reddit-comment offensiveness and workplace-email politeness ratings: 7,210 texts, 43,402 judgments and 971 annotators with self-reported demographics on five-point scales. Models span five open-weight families (Gemma 3, Llama 3.1, Ministral 3, OLMo 3, Qwen 3) from 4B to 70B parameters, with 11 matched base–instruct pairs.
Does demographic conditioning ever help?
Yes, for base (non-instruction-tuned) models: conditioning improved alignment with the target group for 10 of 11 base models and for essentially every demographic group, and the gain doubles under a sharpness-insensitive metric. The failure pattern is specific to instruction-tuned judges, is strongest on offensiveness, and is not fixed by larger model size.
What should practitioners do instead of demographic prompting?
The authors argue demographic prompting should not be treated as a substitute for recruiting annotators from the groups of interest, and warn that an unconditioned LLM judge is not neutral: without a profile, models default toward White, college-educated annotator perspectives.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure-region text (axis labels and in-figure annotations of Figures 2–6 and 10–13) is interleaved and garbled by PDF extraction; figure-level values could not be independently verified, though all numbers cited in this review come from the prose and Tables 1–5, which extracted cleanly.
- No explicit repository URL appears in the extracted text; codeAvailable=true is inferred from Appendix B's statement that prompt configurations are 'released with the code'.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.