Initial Evaluation of Potential Bias in Coverage of Humans in Wikidata
cs.CY · cs.AI · cs.CL
Quick verdict
Audits 6.5M Wikidata humans, finding wide gender, geographic and language gaps; valuable and open, but with denominator inconsistencies and circular baselines.
Summary
The paper presents an open-source auditing platform that ingests 10,128,165 statement bindings for 6,505,428 human entities from Wikidata via the QLever SPARQL engine and audits representation across gender, sexual orientation, citizenship, birthplace urbanicity, ethnicity, multilingual coverage, and occupation. Missingness is classified with Rubin's taxonomy and Little's MCAR test; logistic regressions on a 200,000-row subsample probe MAR versus MNAR; distributions are compared to population baselines with chi-square tests, Wilson intervals, and disparity ratios. Reported results: 28.71% of humans with a stated gender are women; only 38.26% have citizenship statements, with Western Europe and North America at roughly 53% of those; 2.48% of 1.8 million classifiable birthplaces are rural versus a 27.4% global baseline; ethnicity statements fall below 1.2%; labels and descriptions are heavily English-skewed. The authors recommend targeted editing campaigns, rural coordinate enrichment, and multilingual description completion. Acknowledged limitations include a single static snapshot, time-invariant urbanicity classification, and privacy constraints. This review additionally flags apparent numeric inconsistencies — the citizenship count (5,178,210) does not equal 38.26% of 6,505,428; Table 1's assumed shares use an unclear denominator; the abstract's 18.2% non-English description figure conflicts with 88.1% English coverage in Results — plus circular Wikidata-derived baselines, warranting caution on the specific disparity ratios.
Review complete
Assessment at a glance
6
risk flags identified
● high 3
● medium 3
● low 0
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Women make up 28.71% of the 5,222,308 Wikidata humans with a stated sex-or-gender (80.28% coverage), departing from 50% parity (chi-square 948,210.4, Cohen's w = 0.426).
- Only 38.26% of humans carry citizenship statements, with WENA countries at ~53% of those statements despite ~14% of world population (chi-square 1,842,910.1).
- Of 1,843,024 classifiable birthplaces, 2.48% are rural versus a 27.4% global baseline, a disparity the authors score as 0.07x.
- Ethnicity (1.20% coverage) and sexual orientation (0.234% coverage) are the most missing axes and behave as MNAR; gender, citizenship, birthplace, language, and occupation missingness appears MAR.
- Multilingual metadata is severely English-skewed (descriptions: 88.1% English vs 8.4% Hindi, 1.8% Swahili), motivating targeted editing campaigns, rural coordinate enrichment, and Hindi/Bengali/Swahili description completion.
Strengths
Unmatched scale with unusual operational transparency
One unified QLever query captured 10,128,165 bindings over 6,505,428 humans; the pipeline ran in ~8 minutes on a 32 GB desktop and the paper reports energy/carbon (9.61 Wh, 3.70 g CO2e, Methods and Results §3) — rare openness for an audit study.
Missingness treated as a first-class statistical object
Beyond descriptive shares, Rubin's taxonomy, Little's MCAR test, and logistic probes on a 200,000-row subsample distinguish MAR axes (P21, P27, P19, P1412, P106) from MNAR (P91, P172) — a methodological step above prior descriptive Wikidata audits.
Assumption-sensitive sexual orientation analysis
Open and re-runnable
Granular, actionable findings
Weaknesses
Internal numeric inconsistencies in at least four headline results
Circular and mismatched baselines; underived disparity ratios
Null-hypothesis testing is degenerate at this scale
Possible upstream selection bias
Editorial hygiene
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Preliminary Review: Initial Evaluation of Potential Bias in Coverage of Humans in Wikidata (Kronk, 2026)
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What percentage of Wikidata human entries are women?
In this August 2026 audit of 6.5 million human entities, 28.71% of the 5.2 million entries with a sex-or-gender statement were labeled female, 71.27% male, and 0.02% nonbinary or trans; overall statement coverage was 80.28%.
Is Wikidata biased toward Western countries?
Yes, substantially, per this study: only 38.26% of human entries had any citizenship statement, and Western Europe and North America accounted for roughly 53% of those statements while representing about 14% of world population — which the authors classify as severe overrepresentation.
How complete is Wikidata's data on ethnicity and sexual orientation?
Very incomplete: only 1.20% of human entries carried an ethnicity statement and just 0.234% a sexual orientation statement. Statistical tests suggested these two properties are missing not at random (MNAR), meaning omissions are systematic rather than chance.
Can I reproduce this Wikidata bias audit?
The authors state all code and data except the QLever dataset are on GitHub (Superraptor/debias-wikidata). Wikidata itself is public, so the audit can be re-run against a current snapshot — the analysis pipeline reportedly took about 8 minutes on a 32 GB desktop — though numbers will drift as Wikidata is edited.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-5 are referenced but not analyzable from the extracted text; choropleth contents and Figure 4/5 language comparisons are unverified.
- The abstract is truncated mid-sentence ('under-documented') and page-header/copyright boilerplate is interleaved throughout — minor extraction noise, though the extracted numbers appear faithful.
- No supplementary tables are present in the extract, so disparity-ratio derivations could not be checked against any source beyond the prose.
- Placeholder figure references ('Figure X/Y/G/C') may be authorial or extraction artifacts; flagged either way.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.