Can LLMs Normalize Databases? A Benchmark and Multi-Agent Framework for Schema Normalization
Dong-Jae Koh, Huisu Kim, SeongHwan Yoon, Lasse M. Jantsch, Chun-Hee Lee, Seonghyeon Lee, Young-Kyoon Suh
cs.CL
Quick verdict
DNBENCH claims LLMs fail schema normalization; MARS adds 82% relative gain — well-instrumented and candid, but single-run, quantized-only evidence.
Summary
The paper asks whether LLMs can perform end-to-end relational schema normalization. The authors build DNBENCH, 3,275 samples derived from Spider and BIRD databases via a three-stage pipeline (FK-aware subsampling, discovery plus synthetic violation injection, chain-rule labeling) covering 1NF–BCNF violations with gold decompositions, validated by three experts. A three-axis protocol scores lossless join (a hard gate), structural match (Column/PK F1, FK score), and logical quality (LLM judge plus violation F1), combined into a DNB-SCORE. Four open-weight quantized models (Llama 3.3 70B, Gemma3 27B, Qwen3-30B, Mixtral 8x7B) are tested in Single, Complex, and Real World settings. Average DNB-SCORE falls from 0.328 (FDs provided, earliest violation) to 0.188 (FDs inferred); Violation F1 reaches 0.678 while FK scores stay at 0.104–0.174, so FD inference and inter-table constraint reconstruction are the bottlenecks. MARS, a four-agent pipeline (evidence, diagnosis, generation, deterministic verification with repair), raises Real World DNB-SCORE from 0.253 to 0.423 (zero-shot) and 0.209 to 0.418 (few-shot) with Qwen3-30B — an 82.0% average relative gain over the single-prompt baseline — and outperforms the Miffie refinement baseline. Limitations: single-run results without variance estimates; only quantized open-weight models; FK reconstruction and BCNF cases remain weak; artifacts not yet released.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- DNBENCH (3,275 samples from Spider and BIRD) is the first comprehensive benchmark for end-to-end LLM database normalization covering 1NF-BCNF with gold violation labels and decompositions.
- FD inference is the main bottleneck: average DNB-SCORE falls from 0.328 (Single, FDs provided) to 0.188 (Real World, FDs inferred) across four open-weight models.
- Current LLMs diagnose violations better than they reconstruct schemas: Violation F1 reaches 0.678 while FK scores stay between 0.104 and 0.174.
- MARS improves Real World DNB-SCORE by 82.0% on average over single-prompt Qwen3-30B (0.253 to 0.423 zero-shot; 0.209 to 0.418 few-shot), also beating the Miffie refinement baseline.
Strengths
Fills a real gap with clear positioning
Table 1 contrasts DNBENCH with RDBNorma, EDNA, NormTab, TABARD and Miffie: prior LLM work targets table cleaning, anomaly detection, or 1NF-3NF refinement without schema-level lossless-join/key/FK checks; DNBENCH covers 1NF-BCNF, multi-violation schemas, and explanation scoring.
Controlled, audited dataset construction
Section 3.1: FK-aware subsampling preserves referential integrity (Algorithm 1); synthesizers compose violations in reverse chain order so all injected violations stay observable; each sample carries single and combined gold decompositions plus expected FK lists; three experts audited the 3,275 samples (Appendix A.4).
Evaluation mixes deterministic checks with a validated judge
Candid, mechanism-level failure analysis
Weaknesses
Single-run results with no uncertainty quantification
Quantized, mid-scale open models only
Gold-decomposition scoring conflates validity with imitation
82.0% headline rests on a low base and unpriced compute
Judge-equivalence overclaim
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks whether LLMs can perform *end-to-end database schema normalization* - inferring functional dependencies (FDs), diagnosing normal-form violations, and emitting decomposed DDL with valid keys - arguing prior work (NormTab, TABARD, Miffie) targets table cleaning, anomaly detection, or 1NF-3NF refinement without schema-level checks. It makes four claims: (1) DNBENCH is the first comprehensive benchmark for LLM-driven normalization covering 1NF-BCNF with gold labels (3,275 samples from Spider and BIRD); (2) a three-axis protocol (semantic/structural/logical) aggregates into DNB-SCORE; (3) baselines reveal failures in FD inference, decomposition, and FK reconstruction; (4) MARS, a four-agent pipeline, improves Real World DNB-SCORE by 82.0% over single-prompt prompting. The scope claims are well supported by the Table 1 positioning; the quantitative claims need the statistical caveats below.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is DNBENCH?
DNBENCH is a benchmark of 3,275 samples for testing whether large language models can normalize relational database schemas from 1NF up to BCNF. It was built by deliberately injecting labeled normal-form violations into schemas taken from the public Spider and BIRD text-to-SQL datasets, and it scores model outputs on lossless-join preservation, structural match to a gold decomposition, and explanation quality, combined into a single DNB-SCORE.
Can LLMs normalize databases according to this paper?
Only partially, and poorly in realistic settings. When functional dependencies are explicitly provided, average DNB-SCORE is modest (0.328 for single violations, 0.285 for full violation chains); when models must infer dependencies themselves, it drops to 0.188. Models detect violations reasonably well (F1 up to 0.678) but reconstruct foreign keys poorly (scores of 0.10-0.17).
What is MARS and how much does it improve results?
MARS (Multi-Agent Reasoning for Schemas) splits normalization into four LLM agents - evidence extraction, violation diagnosis, schema generation and deterministic verification - with up to two repair rounds. Using Qwen3-30B it raises Real World DNB-SCORE from 0.253 to 0.423 (zero-shot) and 0.209 to 0.418 (few-shot), an average relative gain of 82.0% over single-prompt prompting, and it beats the Miffie self-refinement baseline.
Is DNBENCH's data or code available?
Not yet. The paper states that all artifacts will be released upon acceptance, so as of this preprint (arXiv 2609.11141, September 2026) neither the benchmark samples nor the evaluation code are publicly downloadable.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The DNB-SCORE formula is extracted as 'Semantic x pStructural x Logical'; the 'p' prefix appears to be a flattened radical, so the aggregation exponent was inferred from the prose (BLEU-inspired geometric mean) rather than from the formula itself.
- The per-table subsampling cap n in Algorithm 1 is never assigned a numeric value anywhere in the provided text; statements about row-sample sizes rely on the few-shot prompt examples only.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.