Language Is an Insufficient Substrate for Quantitative Reasoning, and Consequential Domains Need Large Quantitative Models
Reuben Vandeventer, David Imrem, David J. Wild
cs.AI · cs.LG
Quick verdict
Position paper: LLMs can't ground quantitative decisions; proposes LQM class. Coherent and honest, but formal core elementary and no new experiments.
Summary
This position paper argues that large language models are structurally unfit as the substrate for consequential quantitative decisions — pricing risk, capital allocation, triage, cyber defence — because they are trained on human descriptions of records rather than on the records themselves. The authors formalize lossiness with the data-processing and Fano inequalities (Proposition 1: error is bounded below by the conditional entropy the description discards, independent of model scale) and argue that reproducibility, lineage and calibrated uncertainty likewise fail by construction on a language substrate. Mapping these four properties against model-risk governance regimes (SR 11-7, NAIC, EU AI Act, GMLP; Table 1), they define a distinct class, the Large Quantitative Model: substrate-native, structurally explicit, lineage-complete, uncertainty-calibrated with abstention (Definition 5, Table 2). A sketch instantiation combines lineage-preserving refinement, Mapper-based topological structure with computable inverse images, and conformal calibration, with the LLM confined to the interface. The only empirical evidence is imported from a companion security-operations system (precision 0.91±0.02 on LANL red-team events) that follows this division of labour. Limitations: no new experiments, an elementary formal core, an empirical anchor authored with overlapping personnel, a never-quantified information loss η, and a disclosed conflict of interest — all three authors are affiliated with Duo Dimensio LLC, which builds such systems; instantiation operators are withheld.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Fidelity, reproducibility, lineage and calibrated uncertainty are properties of the training representation and the computed map, so no scale of LLM trained on descriptions can supply them (Sections 1–3, Proposition 1).
- These four properties, which model-risk governance regimes (SR 11-7, NAIC, EU AI Act, GMLP) already demand, define a distinct foundation-class model: the Large Quantitative Model (Definition 5, Table 1).
- An LQM is buildable by refining records into a lineage-complete substrate, building a Mapper-style explicit structure with computable inverse images, and learning calibrated dynamics over it, with the LLM confined to the interface (Section 7).
- A deployed SOC system that confines the LLM to narration while a structured policy decides (precision 0.91±0.02 on LANL red-team events) evidences the division of labour (Section 8).
Strengths
Correctly aimed formal core
Proposition 1 (Section 3.1) chains the data-processing and Fano inequalities to show the error floor from lossy description is independent of parameter count, training budget or inference compute — a clean, verifiable localization of the failure in the representation rather than the model.
Governance-as-specification mapping
Table 1 (Section 4) aligns SR 11-7/OCC 2011-12, the NAIC model bulletin, EU AI Act high-risk obligations and GMLP with the four properties, converting compliance requirements into design requirements — the paper's most original and practically useful move.
Unusual candor about limits
Precise neighbour-class delineation
Weaknesses
No new empirical evidence
Unquantified pivotal quantity
Definitional boundary risk
Asserted taxonomy
Undisclosed instantiation and COI
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is a position paper (arXiv:2609.12105, cs.AI/cs.LG) with no new experiments. The negative claim: fidelity, reproducibility, lineage and calibrated uncertainty are properties of the training representation and the computed map, not of scale, so an LLM trained on descriptions of quantitative records cannot supply them by construction (Sections 1–3). The constructive claim: these four properties define a foundation-class model, the Large Quantitative Model (LQM), with properties P1–P4 — substrate-native, structurally explicit, lineage-complete, uncertainty-calibrated with abstention (Definition 5, Section 6.1). Section 4 grounds the properties in governance regimes (Table 1: SR 11-7/OCC 2011-12, NAIC model bulletin, EU AI Act high-risk obligations, FDA/Health Canada/MHRA GMLP), with the memorable inversion that governance is the specification, not the obstacle. Section 7 sketches one instantiation: refine records into a lineage-complete substrate; build a Mapper-based explicit structure whose vertices are sets of named records (computable inverse images deliver P3 as a by-product); learn dynamics with conformal calibration and abstention; confine the LLM to intent parsing and constrained narration. Section 8 imports evidence from a companion security-operations system.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is a Large Quantitative Model (LQM)?
A model class proposed in this paper: a foundation-class system trained on a domain's own quantitative record (tables, graphs, time series) rather than on text, holding an explicit inspectable structure, a computable lineage map from every output back to source records, and distribution-free calibrated uncertainty with an explicit option to abstain. It is defined by four properties (P1–P4) and positioned against LLMs, tabular foundation models and world models.
Why do the authors claim LLMs cannot do quantitative reasoning?
They argue LLMs are trained on human descriptions of data, and description is a lossy compression of the quantitative record: by the data-processing and Fano inequalities, error is bounded below by the information the description discarded, regardless of model scale. Sampling-based outputs, weight-held representations and generated confidence text also preclude reproducibility, lineage and native calibration.
Does the paper include experiments?
No new ones. It is a position paper; its only empirical evidence is imported from a companion security-operations system (SENTINEL-RL) that achieved precision 0.91±0.02 on held-out red-team events of the 58-day LANL authentication dataset, with the LLM confined to producing analyst-facing narrative while a graph-based policy made the decisions.
Do the authors have a commercial interest in this position?
Yes, disclosed in the paper: all three authors are affiliated with Duo Dimensio LLC, which develops systems of the kind described in the paper's instantiation section; one author is also an Indiana University faculty member. The instantiation's specific operators are not published.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- PDF extraction flattened formal notation: superscripts (e.g., '104–105 hosts', '2.4 × 107 edges' are 10^4–10^5 and 10^7) and conditional bars in H(q(D)|T(D)); values reconstructed from context. Text is otherwise complete through reference [47]; no truncation observed.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.