Large Knowledge Model: From Papers to a Scientific Reasoning Landscape
Yuan Huang, Sihan Hu, Hongyu Gu, Chao Ma, Jiaxing Zhang, Zhiyong Zou, Caiyu Fan, Yan Xiao, et al.
cs.AI · cs.CL · cs.IR
Quick verdict
LKM turns 40M+ papers into reasoning graphs, reporting QA gains up to 14.69 pp; promising, but margins are small, untested, and graph fidelity unmeasured.
Summary
The Large Knowledge Model (LKM) is a scientific knowledge infrastructure that converts research papers into source-grounded reasoning graphs — typed nodes for questions, claims, premises, reasoning chains, methods, and evidence, canonicalized via content hashing. At corpus scale it covers reasoning graphs for over 40 million papers and abstracts for over 70 million, organized into three aligned views: Question, Workflow, and Evidence Landscapes. Vector retrieval over these objects is coupled with graph traversal to support scientific search, evidence-grounded QA, and research planning. Evaluation uses 244 matched PaSaMaster search queries and QA benchmarks with GPT-5.4 fixed as the answering model: LKM Hybrid reaches NDCG@5/10/20 of 22.19/21.00/20.93, beating Science Navigator by 0.75–1.65 points; RAG lifts accuracy over the bare model by 9.30 (ChemBench), 4.20 (PubMedQA), and 14.69 (SciBench) points; and on ScholarQA, graph context improves citation F1 over search-only context by 5.71–6.67 points with confidence intervals excluding zero. Main limitations: graph-extraction fidelity is never evaluated; search and QA gains lack significance testing (e.g., only +0.20 on PubMedQA vs Science Navigator); the planning and workflow capabilities motivating the design go empirically untested; and the fully proprietary system ships no code, limiting reproducibility.
Review complete
Assessment at a glance
5
risk flags identified
● high 2
● medium 2
● low 1
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Papers can be represented as source-grounded typed reasoning graphs whose questions, claims, reasoning chains, and evidence are addressable by both semantic retrieval and structural traversal over the same object identities (Sec 3).
- The deployed system covers reasoning graphs for over 40 million papers and abstract representations for over 70 million (Sec 3.4).
- With GPT-5.4 fixed as the answering model, LKM retrieval raises QA accuracy over the bare model by 9.30, 4.20, and 14.69 pp on ChemBench, PubMedQA, and SciBench, and over Science Navigator by 1.26, 0.20, and 2.28 pp (Sec 5.3).
- Graph-addressable context improves ScholarQA citation F1 by 5.71–6.67 pp over search-only context while keeping answer quality comparable, with paired CIs excluding zero (Sec 5.3).
- The aligned Question, Workflow, and Evidence Landscapes form a substrate for research planning and a cumulative scientific ecosystem (Sec 4, Sec 6 — asserted, not evaluated).
Strengths
Deployed at genuine corpus scale
Reasoning graphs for 40M+ papers and abstract representations for 70M+ (Sec 3.4), exposed via a documented public API (Appendix A: /search, /reasoning/search, /papers/graph, /claims/{id}/reasoning). Few argumentation-graph systems operate anywhere near this scale.
Evaluation design isolates the retrieval contribution
GPT-5.4 is fixed as the answering model across all configurations (Sec 5.1), so the +9.30/+4.20/+14.69 pp ChemBench/PubMedQA/SciBench deltas (Fig 6) are attributable to retrieval rather than model changes.
ScholarQA analysis meets a real statistical standard
Formally specified, falsifiable representation
Weaknesses
The substrate is never validated
Statistical treatment is absent outside ScholarQA
The headline novelty is unevaluated
Proprietary system, under-specified baselines
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
LKM is a system paper from DP Technology, the Institute of Theoretical Physics (CAS), Peking University, and the AI for Science Institute (corresponding authors include Weinan E). It claims three things: (1) a source-grounded reasoning-graph representation of papers (Sec 3, Definitions 1–7) in which questions, claims, ordered reasoning chains, methods, and evidence are typed, addressable objects; (2) a corpus-scale deployment — reasoning graphs for 40M+ papers, abstract representations for 70M+ (Sec 3.4) — organized into three aligned cross-paper views (Question, Workflow, Evidence Landscapes, Sec 4); and (3) measurable gains for scientific search and QA attributable to this representation: NDCG wins over Science Navigator on 244 matched PaSaMaster queries (Fig 5), accuracy gains of +9.30/+4.20/+14.69 pp over a bare GPT-5.4 on ChemBench/PubMedQA/SciBench (Fig 6), and citation-F1 gains of +5.71/+6.67 pp from graph context on ScholarQA-CS/Multi (Fig 7b). The research-planning and self-iteration vision (Sec 6) is aspirational and explicitly not evaluated.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the Large Knowledge Model (LKM)?
LKM is a scientific knowledge infrastructure developed by DP Technology, the Institute of Theoretical Physics (CAS), Peking University and collaborators. It converts research papers into reasoning graphs — structured nodes for questions, claims, premises, reasoning steps, methods, and evidence, each linked back to its source text — and indexes over 40 million papers this way. It supports scientific search, evidence-grounded question answering, and research planning through a public API on the Bohrium platform.
How much does LKM improve scientific QA accuracy?
With GPT-5.4 fixed as the answering model, adding LKM retrieval raised accuracy over the same model without retrieval by 9.30 percentage points on ChemBench (64.78 to 74.08), 4.20 on PubMedQA (77.20 to 81.40), and 14.69 on SciBench (76.21 to 90.90). Against the commercial Science Navigator retrieval baseline the margins were much smaller: +1.26, +0.20, and +2.28 points respectively, and those were not significance-tested.
Is LKM open source? Can I reproduce the results?
No. The paper documents a public API (knowledge search, reasoning-chain search, paper graph, claim reasoning endpoints on Bohrium), but releases no code, extraction prompts, index snapshots, or evaluation harness. Independent reproduction of the reported benchmark numbers is not currently possible; only qualitative probing through the API is.
How is LKM different from GraphRAG or ordinary RAG?
Instead of retrieving generic text chunks, LKM retrieves typed scientific objects — questions, claims, and complete reasoning chains — that retain their argumentative structure and source links, and organizes them into three cross-paper views (Question, Workflow, Evidence Landscapes). The paper reports that this graph context improves citation F1 on ScholarQA by about 6 points over search-only context, while answer quality stays roughly the same.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure values (Figs 5–7) were reconstructed from flattened text extraction; the series-to-condition mapping within each figure could be misread.
- The abstract and some prose are truncated mid-sentence in the extraction, and Appendix A tables are partially garbled.
- The extraction pipeline (models, prompts, human validation) is not described in the available text, which limits methodology assessment beyond what is stated.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.