Generating a Consistent Enterprise: Synthesis and Reference-Free Evaluation of Multi-System Business Data
Benjamin Gruenbaum, Doron Porat, Assaf Natanzon, Roy Zavida, Chen Dinachi, Or Itzahary, Omer Niv
cs.AI
Quick verdict
Claims deterministic synthetic enterprise data with reference-free eval; sound engineering, but evaluation is circular and lacks external validation.
Summary
The paper presents Era by Eon, an industrial system that generates complete fictional enterprises from a five-value specification: industry, company size, business model, application portfolio, and seed. A single entity graph holds every fact; 66 product simulators (Salesforce, Zendesk, Slack, Gong, and others) independently rebuild that graph and project it into vendor-native formats, so cross-system consistency follows from determinism rather than synchronization. Per-subsystem random streams and keyed per-value generators keep output stable as code changes. Realism comes not from training data but from 13 cited reference targets (three marked as estimates), with flaws deliberately planted in 3% of records. Evaluation needs no real counterpart: a five-axis scorecard of 28 checks, a rule-based adversarial detector, and soundness checks including a boosted-tree two-sample test against column-shuffled copies. In a paired experiment over 23 companies (same seeds, legacy versus full realism profile), mean realism rose from 60.3 to 99.1, the worst company from 41.1 to 94.9, and detector flags fell from 55.2% to zero; an unseen-seed CI gate scores worst at 91. A second generator builds question-conditioned databases with planted qualifying rows, near misses, and exact labels. Acknowledged limitations: generator and scorecard share targets (circularity), no external generator comparison, uncalibrated detector, template-based text, and axis means resting on only 4-7 companies.
Review complete
Assessment at a glance
4
risk flags identified
● high 1
● medium 2
● low 1
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- One entity graph, deterministically rebuilt by each of 66 simulators from a five-value specification and seed, yields cross-system consistency without any synchronization or data exchange.
- Realism can be constructed from 13 cited reference statistics and verified reference-free with a fixed five-axis, 28-check scorecard, a rule-based adversarial detector, and soundness checks including a shuffled-copy classifier two-sample test.
- Under these fixed instruments over 23 paired companies, mean realism rose from 60.3 to 99.1, the worst company from 41.1 to 94.9, and detector-flagged records fell from 55.2% to zero, holding on an unseen development seed (worst 91).
- A question-conditioned generator can force qualifying rows and controlled near misses, then compute exact answer labels by executing the questions against the finished tables, with verifiable unanswerable questions.
Strengths
Determinism architecture with contained change propagation
Section 3.2's two-level design — 21 per-subsystem streams seeded as seed⊕constant, plus keyed per-value generators (e.g. call-load/seed/rep) — lets 66 simulators rebuild one consistent company with zero inter-communication and bounds regeneration diffs to changed records and their referencers. The prospect-list example (ten prospects per customer dominating draw counts) concretely justifies the stream boundaries.
Paired, fixed-yardstick evaluation design
Section 7.1: 23 fixed specifications generated twice with identical seeds, only the realism profile toggled; instruments predate tuning; and an unseen-seed CI gate (worst company 91, zero flags, Section 7.1) directly controls seed overfitting. Table 2 honestly discloses which axis means rest on small subsets (F1 n=5, F2 n=5→7, F4 n=4).
Honest provenance and self-criticism
Question-conditioned generator with verifiable ground truth
Weaknesses
Circular evaluation by construction
No external baseline or validation
Statistically fragile axis aggregates
Reproducibility stops at a closed service
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is an industrial systems paper (arXiv:2609.11286v2, cs.AI) describing the data generator behind Era by Eon, a commercial service for synthetic enterprise environments; a companion paper (arXiv:2609.09853) covers the agent benchmark built on top. The motivating setting is genuinely constrained: enterprise data is confidential, so neither the train-on-real-data paradigm of SDV/CTGAN nor reference-based quality scoring applies (Section 1). Four contributions are claimed: (1) a deterministic generator producing one entity graph projected into 66 vendor-native product simulators from a five-value specification (Sections 3-4); (2) a question-conditioned relational database generator with forced qualifying rows, near misses, and exact labels (Section 5); (3) a reference-free evaluation methodology (Section 6); (4) a measured improvement from 60.3 to 99.1 mean realism under fixed instruments (Section 7).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Era by Eon?
It is a hosted service (console.era.eon.io) plus per-product Docker images that generate complete fictional companies - employees, customers, deals, tickets, calls, chats, and documents - from a small specification and a random seed, then serve them through simulators that speak the real REST APIs and MCP interfaces of 66 business products such as Salesforce, Zendesk, Slack, and Gong. It is free for builders and fully synthetic, so no real customer data is involved.
How can you evaluate synthetic data when there is no real dataset to compare against?
The paper uses three reference-free instruments: a scorecard of 28 statistical checks along five axes, compared against 13 cited reference statistics about real business data; a rule-based adversarial detector that hunts for generation artifacts such as numbered name variants and evenly spaced timestamps; and soundness checks including a boosted-tree classifier test against column-shuffled copies of the data and a q-error cardinality measurement.
Does the realism score of 99.1 mean the data is indistinguishable from real enterprise data?
No. The generator and the scorecard read the same reference targets, so high scores largely verify adherence to the declared design rather than fidelity to real data. The paper itself states that a zero detector report 'does not prove that the generated data is indistinguishable from real enterprise data' and that the detector has not been calibrated against real records.
Is the Era by Eon generator open source?
No source code is released. The system is available as a free hosted service and as container images (erabyeon/* on Docker Hub) that run offline; pinning the seed, anchor date, and image tag regenerates the same company byte-for-byte, which is what makes the determinism claims independently checkable.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1 (the entity-graph diagram of Bellwether Industries) is referenced but its visual content cannot be assessed from the extracted text; all narrative claims here rest on the body text and tables, which extracted completely through the reference list.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.