Newer Is Not Fairer: Gender Stereotyping in Text-to-Image AI Across Model Generations
Shesh Narayan Gupta, Nik Bear Brown
cs.CV · cs.AI · cs.CY · cs.LG
Quick verdict
This 8,000-image audit claims occupational gender bias in Stable Diffusion does not improve monotonically across generations (SDXL worst); its large effects are statistically sound, but classifier validation (50 images, one reviewer) and off-native 512px generation for SDXL leave residual doubts.
Summary
The paper audits occupational gender representation across four Stable Diffusion generations (SD 1.5, SD 2.1, SDXL, SD 3 Medium). The authors generate 8,000 images (20 occupations x 5 prompt templates x 20 seeds), classify apparent gender with DeepFace, and compare against 2023 U.S. BLS workforce demographics. Key results: 76.4% of images are male-classified; 57.6% of images for historically female-coded occupations are male; models underrepresent women by 20-46pp versus BLS, with the largest deviations for near-balanced occupations (scientist: 48% female in BLS, 82-99% male in outputs). Bias is non-monotonic across generations: SD 1.5 and SD 2.1 are statistically indistinguishable, SDXL is significantly more skewed, and SD 3 Medium partially recovers; the authors name this a 'deployment gap.' Prompt sensitivity increases across generations, and GPT-image-1 shows a small (Cramer's V = 0.080) bias reduction on a 5-occupation exploratory subset. Main limitations, largely acknowledged: single-reviewer classifier validation on only 50 images, binary gender labels, all open-source models run at 512x512 even though SDXL's native resolution is higher, one GPT prompt with n = 20, and no statistical tests for the racial-composition findings.
Review complete
Assessment at a glance
6
risk flags identified
● high 0
● medium 4
● low 2
4 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Across 8,000 Stable Diffusion images spanning 20 occupations, 76.4% depict male-presenting subjects (BH-adjusted p < 2.2 x 10^-16), and 57.6% of images for historically female-coded occupations depict male subjects.
- Compared with 2023 BLS workforce demographics, the models underrepresent women by 20-46pp on average, with the largest deviations for near gender-balanced occupations such as scientist (48% female in BLS, 82-99% male in outputs).
- Bias does not improve monotonically across model generations: SDXL is significantly more gender-skewed than both SD 1.5 and SD 3 Medium, a pattern the authors term the 'deployment gap.'
- Prompt sensitivity (SD of male% across 5 phrasings) increases across generations, so single-prompt bias audits underestimate the range of a model's demographic outputs.
- A preliminary, exploratory comparison suggests GPT-image-1 exhibits modestly less occupational gender bias than the open-source models (Cramer's V = 0.080, a small effect).
Strengths
Large, controlled, crossed experimental design
4 models x 20 occupations x 5 prompts x 20 seeds = 8,000 images with n = 100 per occupation-model cell; 20 fixed seeds shared across models and occupations (Sec. 3.1, 3.4), enabling like-for-like generational comparison that prior audits (Luccioni et al.; Bianchi et al.) did not perform across four SD versions.
Consistently correct statistical hygiene
Binomial and chi-square tests with Cramer's V effect sizes throughout, BH correction applied across all 10 reported tests (Sec. 3.6, Table 1), and I verified Table 1's chi-square and V values are arithmetically consistent with the reported male percentages at N = 2,000 per model. The one non-significant comparison (SD 1.5 vs SD 2.1, raw p = 0.792) is honestly reported as not surviving correction.
Grounding in external demographic reality, not just a 50/50 baseline
Unusually candid limitation handling
Weaknesses
Thin classifier validation for the measurement instrument
Off-native resolution confounds the cross-generation trajectory
The 'deployment gap' is descriptive framing, not analysis
Racial-composition and prompt-sensitivity findings lack inferential statistics
Occupation grouping blurs 'male-skewed' and 'near-balanced'
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Do newer Stable Diffusion models have less gender bias?
Not necessarily. This study of 8,000 generated occupational images found bias does not improve steadily across generations: SDXL showed significantly more gender skew (81.0% male) than the older SD 1.5 (77.0%) and the newer SD 3 Medium (70.1%). SD 3 Medium was the least biased of the four tested, but the authors warn that widely used older versions like SDXL remain the most biased.
How male-skewed are AI-generated occupational images?
In this study, 76.4% of 8,000 images generated by four Stable Diffusion versions for 20 occupations depicted male-presenting people. Even for historically female-dominated jobs such as nurse, teacher, and hair stylist, 57.6% of generated images showed men. Against U.S. labor statistics, models underrepresented women by 20-46 percentage points on average.
How did the researchers label gender in the generated images?
They used DeepFace, an open-source facial-analysis tool, to classify each image as apparently 'Man' or 'Woman.' A single author manually checked a random sample of 50 images with 96% agreement. The authors acknowledge this validation is small, has no second annotator, uses a binary label, and that DeepFace was trained on real photos rather than AI-generated ones.
What are the main limitations of this study?
The gender classifier was validated on only 50 images by one reviewer; all open-source models were generated at 512x512 even though SDXL's native resolution is higher, which could affect face quality; the GPT-image-1 comparison used only 5 occupations, one prompt, and 20 images each; racial-composition findings had no statistical tests; and the BLS reference data is U.S.-specific (2023).
Is this the first study to compare gender bias across Stable Diffusion versions?
The authors claim it is the first controlled longitudinal comparison of occupational gender stereotyping across four Stable Diffusion generations (SD 1.5, SD 2.1, SDXL, SD 3 Medium) within a single experiment. Prior audits such as Luccioni et al.'s 'Stable Bias' documented bias in Stable Diffusion models but did not run a unified multi-generation comparison.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Only the paper's text was analyzed; figures (heatmaps, bar charts, trajectory plots) could not be inspected, so figure-level claims rest on the reported numbers alone.
- The abstract's 95% CI [75.1%, 78.7%] is inconsistent with the +/-1.8pp margin stated in Sec. 3.1 (which implies [74.6%, 78.2%]); treated as a reporting slip rather than an inferential problem.
- Text appears complete through the references, but no supplementary material (per-cell tables, validation images, seed lists) was available to verify flagged-cell counts or classification details.
- Code repository availability is taken from the paper's own statement (Sec. 7); the repo contents were not checked per the no-network constraint.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.