UniDataAgent: An Ontology-Grounded Agent for Enterprise Question-to-Report Automation
Yutai Duan, Yahui Zhao, Zhangti Li, Yu Ma, Yu Ma, Zhenfeng Qi, Shaoyang Yuan, Jing Fan, et al.
cs.CL
Quick verdict
A deployed ontology-grounded enterprise QA system with strong reported gains (95.0% vs 72.5% RAG), but evaluation is small, single-site, and baseline-thin.
Summary
UniDataAgent is a two-stage enterprise question-to-report system deployed at China Unicom, presented as an ICDM 2026 demo. The offline Ontology Acquisition and Validation (OAV) stage compiles database metadata, business documents, and expert knowledge into a versioned enterprise ontology using an expert-authored Business Skill, constrained candidate generation, and a knowledge-grounded verifier that checks coverage, consistency, provenance, and question answerability, with expert review reserved for high-impact changes. The online Question-to-Report Execution (QRE) stage grounds each question in a retrieved semantic contract (metric calibers, physical mappings, query constraints, lineage), runs a ReAct-style loop over read-only SQL and sandboxed tools with per-observation validation gates, and synthesizes evidence-cited reports. Across 27 enterprise tables, the authors report ontology construction in a few hours versus roughly one week manually, and a roughly 200-second end-to-end demonstration. On 40 real enterprise questions, strict accuracy is 95.0% versus 72.5% for document RAG, with the largest gains on complete regional comparison (11 questions, 100.0% vs 63.6%) and cross-metric diagnosis (7 questions, 100.0% vs 57.1%). Main limitations: a single enterprise, a single LLM (DeepSeek-V4), one baseline, no component ablation, no statistical testing, internally authored reference answers, and no code or data release, so the headline numbers are not independently verifiable.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 19 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Separating offline ontology acquisition (OAV) from online execution (QRE) through a typed ontology contract enables reusable, semantically faithful enterprise question-to-report automation.
- LLM-assisted ontology construction with constrained generation and question-based verification reduces build time from about one week to a few hours across 27 enterprise tables and roughly thousands of metric types.
- Ontology grounding achieves 95.0% strict accuracy on 40 real business questions versus 72.5% for document RAG, with the largest gains on complete regional comparison (11 q) and cross-metric diagnosis (7 q).
- Per-observation evidence gates that check caliber, scope, and provenance allow factual, citation-anchored reports with interpretations explicitly marked as analysis.
- The system is deployed at China Unicom with claimed cost savings and potential cross-enterprise replicability.
Strengths
Clear offline/online separation via a typed ontology contract
OAV persists a versioned ontology while QRE consumes a compact semantic contract encoding metric calibers, physical mappings, query constraints and lineage (Sec. II), directly targeting the caliber/scope drift the introduction attributes to plain Text-to-SQL.
Verification-first construction loop with expert governance
OAV's staged loop (Business-Skill grounding, semantic plan, plan assurance, constrained generation) plus a question-answerability verifier that triggers repair or replanning, with expert approval reserved for high-impact changes (Sec. II-A, Fig. 2), addresses provenance rather than relying on one-shot LLM extraction.
Task-type-stratified evaluation aligned with the thesis
Evidence-constrained synthesis discipline
Real production deployment evidence
Weaknesses
Single weak baseline; central prior art never compared
No component ablation
Small-sample evaluation with no statistics
Evaluation circularity risk
Unverifiable efficiency, cost, and generality claims
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
UniDataAgent — Preliminary Review
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is UniDataAgent?
UniDataAgent is an ontology-grounded enterprise data agent from China Unicom Software Research Institute (with Nankai University), presented as an ICDM 2026 demo. It turns natural-language business questions into evidence-linked analytical reports in two stages: an offline OAV stage that builds a versioned enterprise ontology from metadata and business documents, and an online QRE stage that retrieves semantic contracts, runs a ReAct-style agent over read-only SQL and sandboxed tools, and validates every observation before it enters the report.
How accurate is UniDataAgent according to the paper?
On 40 real enterprise questions from China Unicom, the paper reports 95.0% strict answer accuracy versus 72.5% for a document-RAG baseline, including perfect scores on the complete regional comparison subset (11 questions) and the cross-metric diagnosis subset (7 questions). Caveats: the evaluation is small (40 questions), single-company, single-LLM (DeepSeek-V4), has no statistical testing, and only one baseline.
Is UniDataAgent's code or data publicly available?
No. The evaluation uses private enterprise tables (27 tables, thousands of metric types) and the demonstration data were irreversibly transformed; the paper announces no public code, ontology, or benchmark release, so the reported numbers cannot currently be reproduced independently.
How does UniDataAgent differ from Text-to-SQL?
The authors argue that executable SQL is insufficient: enterprise reports must respect organization-specific metric calibers, scopes, and procedures. UniDataAgent therefore grounds each question in a pre-built, expert-reviewed ontology contract (metric definitions, physical mappings, constraints, lineage) instead of inferring semantics per query, and validates intermediate results via evidence gates. Notably, it was benchmarked only against document RAG, not against a Text-to-SQL system.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-5 are available only as captions; the ontology structure, interface screenshots, and demo report visuals could not be assessed.
- A referenced demonstration video (YouTube) and any supplementary material were not accessible; evaluation question lists, checklists, and judging protocol details are absent from the text.
- A few sentences appear truncated or garbled in extraction (e.g., the OAV plan-checking sentence ending before candidate generat...), though Table I and all section headers are intact.
- The paper is a 4-page ICDM 2026 demo-track paper; evaluation depth should be judged against that venue's scope.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.