WiCleanData: Guaranteeing the Type Consistency of Wikidata by Taxonomy Refinement and Constraint Enforcement
Yiwen Peng (IP Paris), Marc Jeanmougin (IP Paris), Thomas Bonald (IP Paris)
cs.AI
Quick verdict
Claims a type-violation-free 420M-fact Wikidata via LLM taxonomy cleaning; impressive scale and open code, but 'better on all tasks' overstates Table 2.
Summary
The paper introduces WiCleanData, a refined version of Wikidata produced by an automated three-stage pipeline that guarantees type-consistent facts. Stage one cleans the taxonomy: classes are detected heuristically, metaclasses and instance-less classes removed, and every subclassOf link judged symmetrically by three small LLMs (Mistral-Small-24B, Qwen3-32B, Gemma-3-27B) under majority voting and confidence thresholding, yielding confirmed/equivalent/reversed/irrelevant decisions that drive link cutting, class merging, and pruning. Stage two simplifies each property's type constraints via hierarchical clustering and lowest-common-ancestor selection with an instance-inflation cap, plus orphan-type removal. Stage three retypes instances and filters facts, retaining ~50M entities and 420M facts (93% of candidates). Intrinsic results show the taxonomy shrinking from 4.2M to 35K classes, zero taxonomic loops and redundant links, and a modest robustness drop (0.80 to 0.75). Extrinsic evaluation on the KGrEaT framework shows gains in document similarity and classification under the all-entities setting, but clustering ARI/NMI for known entities roughly halve (0.197 to 0.114; 0.229 to 0.115) and recommendation F1 stays at floor (0.013), so the claim of comparable-or-better on all tasks overstates Table 2. No significance tests or ablations are reported, though code, parameters, and a public SPARQL interface are provided.
Review complete
Assessment at a glance
4
risk flags identified
● high 1
● medium 3
● low 0
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- An automated three-stage pipeline (LLM-assisted taxonomy refinement, hierarchical constraint aggregation, constraint-based fact filtering) can transform Wikidata into a knowledge graph free of type-constraint violations while retaining ~50M entities and 420M facts (93% of candidates).
- Symmetric LLM link checking with three-model majority voting reduces the taxonomy from 4.2M to 35K classes while nearly preserving taxonomic robustness (0.80→0.75).
- Per-property type constraints can be simplified to fewer, non-redundant types via clustering plus LCA cover selection with an instance-inflation cap (τ=0.02).
- WiCleanData performs comparably or better than Wikidata across downstream tasks in the KGrEaT framework — a claim partially contradicted by the paper's own Table 2 clustering results.
Strengths
Unprecedented joint scope at full Wikidata scale
First pipeline (§3) to clean taxonomy, constraints, and facts together at 50M-entity/450M-fact scale, targeting the ~6M type-constraint violations the introduction cites; prior work ([22],[34]) handled ABox or taxonomy alone.
Principled symmetric LLM link checking
§3.1 queries each subclassOf link in both directions so confirmed/equivalent/reversed/irrelevant follows from paired answers, with three-model majority voting (Mistral-Small-24B, Qwen3-32B, Gemma-3-27B) and token-probability confidence at θ=0.5, explicitly citing bias mitigation [7].
Novel constraint-simplification formulation
Strong reproducibility artifacts
Extrinsic evaluation beyond self-referential metrics
Weaknesses
Extrinsic overclaim contradicted by own results
Intrinsic metrics partly true by construction
No ground-truth validation of LLM decisions
No statistical rigor
Missing baselines and uncharacterized fact loss
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
WiCleanData (CIKM '26 short paper, 5 pages) attacks a well-documented Wikidata pain point. The introduction cites ~6M type-constraint violations, redundant and looping taxonomy paths (film subclassing both visual work and condition, Figure 1a), and constraint type lists polluted by metaclasses, redundant subclasses, and orphan types; the motivating example (Max the dog's *notable work* fact on P800) shows constraints flagging valid facts. Three central claims: (1) a fully automated three-stage pipeline — LLM-assisted taxonomy refinement (§3.1), type-constraint simplification via hierarchical aggregation (§3.2), and constraint-based fact filtering (§3.3) — yields a knowledge graph 'free from any type violation'; (2) the cleaned taxonomy is dramatically simpler while nearly preserving robustness; (3) the result matches or improves Wikidata on downstream tasks. Claims (1)–(2) are supported — largely by construction — via Table 1; claim (3) is partially contradicted by the authors' own Table 2.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is WiCleanData?
WiCleanData is a cleaned, publicly available version of Wikidata built by an automated pipeline: an LLM-verified class taxonomy, simplified per-property type constraints, and facts filtered so that none violate type constraints. It contains about 50 million entities and 420 million facts and is explorable through a web interface with a SPARQL query endpoint.
How many type constraint violations does Wikidata have?
The paper reports roughly 6 million type-constraint violations in Wikidata, and notes many flagged facts are actually valid because the constraints themselves are incomplete or redundant (for example, a dog's 'notable work' fact was flagged). WiCleanData removes violations by cleaning the taxonomy and constraints rather than by editing individual facts.
Does WiCleanData perform better than Wikidata on downstream tasks?
Mixed. On the KGrEaT benchmark it improves document similarity and classification accuracy when all entities are allowed, but clustering quality for known entities roughly halves (ARI 0.197→0.114, NMI 0.229→0.115) and recommendation F1 is unchanged at 0.013. The paper's 'better on all tasks' phrasing is stronger than its own table supports.
Is the WiCleanData code and data available?
Yes. The source code is on GitHub (github.com/peng-yiwen/wicleanData), all pipeline parameters are stated in the paper, and the resulting graph is served at wicleandata.r2.enst.fr with a SPARQL endpoint plus tools to compare the taxonomy with Wikidata's and to explain how each property's type constraints were derived.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1's extracted text is scrambled across panels (a)/(b); panel-specific visual claims were judged from captions and prose only.
- The LLM prompt template and the clustering distance definition are not included in the 5-page text, so exact reimplementation rests on the released code, which was not inspected here.
- The public web interface and GitHub repository were not independently accessed (no network in this review); their contents are taken from the paper's footnotes.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.