DianShi-RxnDB: A Large-Scale, Fine-Grained Organic Reaction Data Platform Built via a Fully Automated Pipeline for Researchers and AI Agents
Yubin Wang, Xingjian Wei, Jiang Wu, Yinfan Wang, Boyu Zhu, Lin Zhang, Jianing Yu, Huazheng Zeng, et al.
cs.CL
Quick verdict
Claims 24M-reaction patent database with 92.95% field accuracy, edging Pistachio; credible but pipeline details deferred and no downstream benchmark shown.
Summary
DianShi-RxnDB is a resource paper describing a platform of ~24 million single-step organic reaction instances (23,999,236) extracted fully automatically from ~1.58 million USPTO/EPO organic-synthesis patents (1976–2025) using DeepSeek-V3-0324 for experimental text and MinerU.Chem for images and schemes; 61.7% (14,808,205) pass automated qualification based on RXNMapper atom mapping and conservation checks, and both qualified and non-qualified records are retained with paragraph-level provenance links. A manual evaluation of 1,300 randomly sampled qualified instances across five fields (Yield, Reactant, Reagent, Catalyst, Solvent) gave a micro-averaged field-level accuracy of 92.95%, ranging from 84.62% (Reagent) to 97.31% (Catalyst); recurring errors involve workup leakage, role assignment, cross-paragraph omissions, and step boundaries. A matched comparison with commercial Pistachio on 100 sampled US patents found 1.368x more records retained after reactant-product deduplication (4,093 vs 2,992) and higher exact agreement against source-grounded references on all six fields across 660 paragraph-matched pairs (largest gap: Yield, +18.33%, partly a representation-policy artifact). The platform offers a Web workbench and an MCP service for AI agents, free for non-commercial use. Main limitations: pipeline technical details are deferred to a future report, no code or bulk-download release is stated, comparison gaps lack statistical testing, there is no recall estimate, and no downstream modeling benchmark demonstrates AI4Chem utility.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A fully automated extraction and normalization pipeline converts ~1.58 million USPTO/EPO organic-synthesis patents into ~24 million fine-grained, provenance-linked single-step Reaction Instances, of which 61.7% (14,808,205) pass automated qualification.
- Manual evaluation of 1,300 randomly sampled qualified instances yields 92.95% micro-averaged field-level accuracy across the Yield, Reactant, Reagent, Catalyst, and Solvent fields.
- In matched comparison, DianShi-RxnDB retains 1.368x more records after reactant-product deduplication than Pistachio (100-patent sample: 4,093 vs 2,992) and achieves higher source-anchored exact agreement on all six evaluated fields (660 paragraph-matched pairs), with the Yield gap partly attributable to representation policy.
- A shared data foundation simultaneously serves researchers (Web workbench with search, comparison, and source verification) and AI agents (MCP tools for multi-step structured retrieval).
Strengths
Rigorous, honestly reported internal quality evaluation
Section 2.5/Appendix C: 1,300 records randomly sampled from the full 14,808,205-instance qualified population, two independent annotators plus third-annotator majority vote on disagreement, per-field results disclosed including the weakest field (Reagent 84.62%) alongside a concrete four-category error taxonomy (workup leakage, duplication/role errors, cross-paragraph omissions, step-boundary errors).
Unusually careful external comparison protocol
Section 2.6/Appendix D: matched 100-patent sample, documented reactant-product dedup key ignoring agent sections (D.2.3), source-paragraph anchoring for 660 pairs, and explicit acknowledgment that the +18.33% Yield gap (99.39% vs 81.06%, Table 9) is partly a representation-policy artifact rather than a pure extraction win.
Paragraph-level provenance throughout
Transparent scale and counting conventions
Dual interface design with a worked agent example
Weaknesses
Core method is not in the paper
No statistical treatment of any comparison
LLM-as-judge without validation
38.3% of the database is quality-unquantified
No downstream benchmark despite AI4Chem positioning
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
DianShi-RxnDB is a resource report for a large organic-reaction data platform rather than an algorithms paper. Starting from 20,256,438 USPTO/EPO patent records, the authors filter by IPC codes, deduplicate within and across sources, and reduce the pool to 1,580,939 full-text documents (Table 2). A fully automated pipeline (DeepSeek-V3-0324 for experimental text on Ascend 910C; MinerU.Chem for images and reaction schemes) produces 23,999,236 single-step Reaction Instances, of which 14,808,205 (61.7%) pass automated qualification based on RXNMapper atom mapping, atom-conservation, and rule checks (Table 5); non-qualified records are retained with their provenance. Central claims: (1) 92.95% micro-averaged field-level accuracy on five manually evaluated fields (Table 6); (2) superiority over Pistachio in deduplicated record count (1.368x, Table 7), representation granularity (Table 8/D2), and field-level exact agreement on all six compared fields (Table 9); (3) a dual Web + MCP interface serving researchers and agents (Sections 3-5). The data snapshot is fixed at 2026-06-25 (Appendix A).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is DianShi-RxnDB and how large is it?
It is an online platform of structured single-step organic reaction records extracted fully automatically from about 1.58 million USPTO and EPO organic-synthesis patents published 1976-2025. It contains roughly 24 million reaction instances, of which about 14.8 million (61.7%) pass automated quality checks; every record links back to the relevant paragraph of its source patent.
How accurate is the data in DianShi-RxnDB?
The authors manually checked 1,300 randomly sampled qualified records against their source patents. Micro-averaged accuracy across five fields (yield, reactant, reagent, catalyst, solvent) was 92.95%, with reagent the weakest field at 84.62% and catalyst the strongest at 97.31%. These estimates apply only to qualified instances, which are 61.7% of the database; the remaining 38.3% were not manually evaluated.
How does DianShi-RxnDB compare with Pistachio or the USPTO datasets?
On a matched sample of 100 US patents it kept 1.368 times more records after deduplication than Pistachio (4,093 vs 2,992), and on 660 paragraph-matched reaction pairs its exact agreement with the patent text was higher on all six compared fields; the largest gap was yield (+18.33%), partly because the two datasets record yields differently. Unlike the freely downloadable USPTO-Lowe dataset, access is via web interface and MCP service only, with no bulk download stated.
Is DianShi-RxnDB free to use?
Web and MCP access is free for non-commercial use; commercial use requires a separate license. The paper does not announce any code release or downloadable dataset snapshot, and the extraction pipeline's technical details are deferred to a future report.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The task metadata lists 8 authors, while the full byline inside the extracted text lists 33 authors plus equal-contribution and correspondence notes — a metadata mismatch, not extraction truncation.
- The provided text ends cleanly after Appendix D.5 with complete sentences; no end-of-document truncation was observed. Appendix headers B.1 and B.3 are present with their tables.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.