Ascent: An Agentic System over the Model Context Protocol for Real-World Clinical Data Analysis
Angelo Ziletti, Leonardo D'Ambrosi, Melanie Tuchardt, Tim Kondziella
cs.AI · cs.IR
Quick verdict
MCP-based clinical agent + EpiTrap error benchmark: capable models gain +27/+20 pp over fixed pipeline, but no compute-matched baseline and judge bias remain.
Summary
ASCENT is a deployed agentic system at Bayer that exposes medical coding, epidemiological question answering, and cohort construction as 31 MCP tools over governed Snowflake data, supporting both OMOP-CDM and native vendor schemas. The authors introduce EpiTrap, a 156-item (native Optum EHR) and 153-item (OMOP) evaluation set where each question is designed around one recognized pharmacoepidemiological error (unit harmonization, prevalence–incidence conflation, denominator ineligibility) and graded by trap-specific rubrics via two independent LLM judges with a third adjudicating. Against a fixed mechanical pipeline (50% native / 49% OMOP accuracy), agentic configurations with capable models (Claude Sonnet 5, DeepSeek-V4-Pro) average +26.7 pp native (95% CI 18.7–35.0) and +20.4 pp OMOP (95% CI 13.3–27.5) across ten configurations, at several times the tool calls and longer runtimes; gpt-oss-120b shows no reliable gain. Reflection prompts add ~27% tool calls and ~13% runtime without consistent benefit; withholding the specialized SQL tools leaves accuracy broadly unchanged. Production logs (36 users, 13,002 calls) document failure modes such as plausible answers to the wrong question. Limitations: single data source, one error per item (accuracy upper-bounds end-to-end correctness), no compute-matched baseline, LLM-only grading with moderate native agreement (κ=0.62), and reliance on proprietary models and data.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Exposing medical coding, epidemiological QA, and cohort tools through a shared 31-tool MCP surface decouples clients, models, and orchestrators from governed data access across OMOP and native schemas (§3).
- With capable models (Claude Sonnet 5, DeepSeek-V4-Pro), agentic configurations beat the fixed pipeline by an average of 26.7 pp native (95% CI 18.7–35.0) and 20.4 pp OMOP (95% CI 13.3–27.5) on EpiTrap (§5.2).
- The gains are model-dependent: gpt-oss-120b shows no reliable improvement over the mechanical pipeline (§5.2).
- Reflection prompts increase tool calls by 27% and runtime by 13% on average without consistent accuracy gains, and specialized SQL tooling reduces calls without improving accuracy for a frontier model (§5.2).
- Real deployment shows the consequential errors are silent — execution success is not validation, and epidemiological plausibility becomes the effective gate (§6).
Strengths
Error-first benchmark design (§4)
EpiTrap scores each question against one designed-in epidemiological error with a trap-specific rubric, independent of solution path — no reference SQL needed. Grounded in epidemiological literature (Ray 2003; Rothman et al. 2008; Horwitz & Feinstein 1980), it is reusable by any text-to-SQL or agentic system on RWD.
Real production evidence (§6)
36 users made 13,002 calls over ~10 weeks across LibreChat, Cursor, Claude Code and other clients; the paper extracts a concrete taxonomy of silent failure modes (plausible answers to the wrong question, silent code-list scope drift, sparse-data subset bias) that execution success alone cannot detect.
Systematic, uncertainty-aware evaluation grid (§5)
Governed, portable architecture (§3)
Candid limitations section
Weaknesses
Baseline not compute-matched (§5.1, admitted)
Judge circularity and moderate agreement (§4)
Eval authored by the system builders (§4)
Generator–schema confound (§3.2)
Accuracy is an upper bound (§4)
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
ASCENT (Bayer AG) is a deployed agentic system for expert-supervised epidemiological analysis of real-world clinical data (§1, §3). It exposes 31 MCP tools in five capability groups — discovery, medical coding, question answering, cohorts, literature — over governed Snowflake data, serving both OMOP-CDM and native vendor schemas. Three contributions are claimed: the governed MCP tool surface, the EpiTrap error-first benchmark (§4), and a model × orchestrator evaluation (§5) plus deployment lessons (§6). The headline claim — agents beat a fixed pipeline by an average of 26.7 pp native and 20.4 pp OMOP — is properly conditioned on 'capable models' (Sonnet 5, DeepSeek-V4-Pro); gpt-oss-120b shows no reliable gain. The abstract rounds these to 27/20 pp, which is acceptable.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is EpiTrap and how is it scored?
EpiTrap is a publicly released evaluation set for clinical text-to-SQL and agentic systems: 156 questions on a native Optum EHR schema and 153 on OMOP-CDM. Each question is designed to tempt a system into one recognized pharmacoepidemiological error — e.g., applying a glucose threshold without unit harmonization, or counting prevalent cases as incident. A response passes if its reasoning avoids the designated error, whatever SQL or tool path it takes; scoring uses trap-specific rubrics applied by two LLM judges with a third model adjudicating disagreements.
How much do agents improve over the fixed pipeline?
On average across ten configurations using Claude Sonnet 5 or DeepSeek-V4-Pro over three orchestrators (Claude Agent SDK, LangGraph, Strands), agents improved accuracy by 26.7 percentage points on the native schema (95% CI 18.7–35.0) and 20.4 points on OMOP (95% CI 13.3–27.5), from baselines of 50% and 49%. The open-weights gpt-oss-120b showed no reliable gain. The improvement costs several times more tool calls and longer runtimes.
Is the code and dataset available?
Yes — the paper states that both the ASCENT code and the EpiTrap questions and rubrics are publicly released on GitHub (bayer-group/ascent-epi-demo). However, the evaluation itself ran on proprietary, de-identified Optum EHR data in Snowflake using commercial frontier models, so the exact reported numbers cannot be replicated without access to that data and those models.
What are the paper's main limitations?
The authors list them explicitly: a single real-world data source; one designed-in error per item, so reported accuracy is an upper bound on end-to-end correctness; the fixed pipeline is not compute-matched to the agents; grading is done by LLMs rather than human experts (native-arm judge agreement is moderate, κ=0.62); and the real-world usage findings are preliminary and qualitative. The system also does not estimate causal effects.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Table 2's interleaved native/OMOP columns extracted imperfectly; per-cell values cited here were parsed with possible misalignment, so recomputed configuration means could differ from the reported 26.7/20.4 averages by roughly a point.
- Figures 1–2 are only partially recoverable from captions; the exact composition of the 31-tool list and the example-session content cannot be verified from the extracted text.
- The evaluation's dependence on proprietary Optum data and unreleased judge prompts means reproducibility is judged from the stated code/data release, not verified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.