The Role of Implicit and Explicit Demographic Signals in Large Language Model-based Student Assessment
Donya Rooein, Luca Benedetto, Dirk Hovy
cs.CL · cs.AI · cs.CY
Quick verdict
192k-response study: 6 LLMs shift scores, feedback, and answers on demographic cues; solid design, but implicit-condition confounds temper the conclusions.
Summary
The paper tests whether demographic information about a student — stated explicitly in a prompt or implied by prior conversation history — changes how six LLMs perform three educational tasks: automated essay scoring on 40 Persuade 2.0 essays, formative feedback on the same essays, and metalinguistic question answering on 40 ELQA items. Using 200 real user profiles and 10-turn prompt histories from the AI Gap dataset, the authors run 192,480 temperature-0 inference calls across GPT-5-mini, GPT-5-nano, Llama 3.3 70B, Llama 3.2 3B, Qwen3 4B and Qwen3 30B, analyzing score deviations, readability (ARI/FRE), length, age of acquisition, sentiment and BERTScore via Lasso regression, per-item Kolmogorov–Smirnov tests with Bonferroni correction, and paired t-tests. Explicit education/SES cues reliably shift readability (higher education leads to less readable answers) and sentiment; implicit conditioning destabilizes AES scoring most for Llama-70B (+1.57 points vs its own default, p<0.001), while GPT models stay stable. Limitations include only 40 items per task, heuristic vote-count significance thresholds (N=10 explicit vs N=5 implicit), topic–demographic confounds in the histories, GPT-5-nano-simulated history responses, and regression coefficients reported without p-values or confidence intervals.
Review complete
Assessment at a glance
6
risk flags identified
● high 0
● medium 3
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- LLMs pick up demographic cues from both explicit persona text and implicit conversation history and change scoring, feedback, and answers accordingly.
- Explicitly mentioned education levels reliably shift response readability (higher education → less readable responses) across most of the six models.
- Implicit demographic conditioning produces larger, less predictable effects, most starkly a +1.57-point AES score inflation for Llama 3.3 70B relative to its own default (p<0.001).
- Lower-education users receive metalinguistic QA answers with lower sentiment, visible under explicit conditioning and, more weakly, implicit conditioning (Qwen-30B).
- Demographic effects are strongly task-dependent: AES scoring is largely stable for most models, while open-ended tasks show systematic readability, sentiment, and length shifts.
Strengths
Controlled counterfactual design at real scale
192,480 temperature-0 inference calls; task input held fixed while only demographic context varies (§2.3), enabling within-model comparisons across six LLMs. Table 1 deviations pass internal arithmetic consistency (Exp-vs-default equals Exp-vs-human minus Model-vs-human within rounding).
Realistic conditioning data instead of synthetic personas
25 attributes and genuine 10-turn prompt histories from 200 AI Gap user profiles (§2.1), avoiding the name-swap persona paradigm; topic modeling (Fig. 2, §E.1) explicitly probes demographic–topic coupling in the histories.
Multi-task, multi-metric coverage matched to RQ2
Statistical care on dependence and skewed attributes
Honest, deployment-relevant framing
Weaknesses
Regression 'significance' without inference
Asymmetric heuristic vote-count thresholds
Confound in the implicit condition
Small, single-sample evaluation
Reporting inconsistencies
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks whether demographic context — explicit persona text (Exp) or implicit conversation history (Imp) — changes LLM behavior on three educational tasks: Automated Essay Scoring (AES) on 40 Persuade 2.0 essays with human expert anchors (1–6 rubric), Formative Feedback (FF) on the same essays, and Metalinguistic QA on 40 ELQA questions. Six models (GPT-5-mini, GPT-5-nano, Llama 3.3 70B, Llama 3.2 3B, Qwen3 4B, Qwen3 30B) × 200 real AI Gap profiles × 25 attributes yield 192,480 temperature-0 inference calls. Core claims: (1) models pick up demographic cues in both conditions; (2) explicit education/SES cues shift readability and sentiment in the pedagogically 'correct' direction; (3) implicit cues produce larger, less predictable effects, most visibly a +1.57-point score inflation for Llama-70B relative to its own default (p<0.001, Table 1). The framing as a controlled counterfactual sensitivity test — profiles are explicitly not the essay authors' true demographics (§2.3) — is correct and clearly stated.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Do LLMs discriminate against students by demographics when grading essays?
In this study, explicitly stated demographic personas caused only small, mostly non-significant score shifts for most models, and GPT models stayed stable. The largest instability came from implicit conditioning via chat history: Llama 3.3 70B scored essays +1.57 points higher than its own default (p<0.001), though still 0.60 points below human raters on the 1–6 scale.
Do LLMs adapt the readability of their feedback to a student's education level?
Yes, when education is explicitly stated in the prompt: four of six models produced less readable responses for higher education levels (e.g., ARI +0.73 per education step for Llama 3.3 70B). Under implicit cues from conversation history, this adaptation largely disappears and effects become model-specific and unpredictable.
What data and models were used in this study?
40 essays from Persuade 2.0 (with expert scores) for automated essay scoring and formative feedback, 40 metalinguistic questions from ELQA, and 200 real user profiles with 25 demographic attributes from the AI Gap dataset. Six LLMs were tested — GPT-5-mini, GPT-5-nano, Llama 3.3 70B, Llama 3.2 3B, Qwen3 4B, Qwen3 30B — in 192,480 temperature-0 inference calls.
Is the code and data for this paper available?
Yes. The paper states that all code and data are publicly available under the GNU General Public License v3.0 (footnote 1), and the underlying datasets (Persuade 2.0, ELQA, AI Gap) are themselves public.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Table headers were garbled in extraction (e.g., Table 1 column labels were reconstructed arithmetically); individual cells may carry minor transcription noise.
- Figures 2–5 are available only as caption/prose text; bubble-chart values and the Imp-vs-Exp correlations (0.56, 0.51, 1.20) are taken from the prose, not the plots.
- Prompt templates in §B appear partially compressed in extraction; exact task-prompt wording could not be fully verified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.