LongAgent: History-Guided Agentic Search for Longitudinal Outcome Prediction
Siyao Wang, Florian Guitton, Shuojie Fu, Guanyu Tao, Kai Sun, Wenjia Bai
cs.AI
Quick verdict
LLM agent searches variable/window/aggregation combos for longitudinal prediction; synthetic gains are marginal (ΔRMSE 0.015, p=0.027) and clinical R²≈0.02.
Summary
The paper introduces LongAgent, an LLM-based agent that automates feature construction for longitudinal outcome prediction in medicine. Instead of hand-crafted predictors, the agent searches a budgeted space of candidates defined by a variable subset (up to ten variables), an outcome-relative time window, and a longitudinal aggregation function (mean, median, p95, p05). At each step it chooses among add/drop/replace/update/stop actions, guided by four evidence sources: search status, candidate history with losses, a coverage summary of evaluated combinations, and transition evidence linking past moves to loss changes. Decisions are made in two LLM stages, and an elastic net evaluator scores each candidate via five-fold cross-validation RMSE. On 360 synthetic datasets (six outcome-generation settings, 500 subjects, 100 variables), LongAgent attains mean RMSE 1.7376, beating the strongest baseline, rule-based search (1.7527), by 0.0151 (95% CI [0.0045, 0.0260]; Wilcoxon p=0.0273). An ablation on 30 datasets shows transition evidence significantly helps (p=0.0488) while coverage summary does not (p=0.3223). On the real IDEA-FAST wearable dataset (1,084 participants, fatigue z-scores), LongAgent is essentially tied with the best baseline (RMSE 0.9937 vs 0.9939), with R²≈0.02 indicating minimal predictive signal overall. Main limitations: tiny effect sizes, a single shared window/aggregation per candidate, no code release statement, and ablations on only a small subset.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 4
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- LongAgent formulates longitudinal predictor discovery as a budgeted search over variable subsets, outcome-relative time windows, and aggregation functions, guided by a four-source history memory.
- On 360 synthetic datasets, LongAgent achieves the lowest mean RMSE (1.7376), beating rule-based search by 0.0151 (95% CI [0.0045, 0.0260]; p=0.0273).
- Adding transition evidence to the agent history significantly improves search performance (p=0.0488 on 30 datasets), while coverage summary alone does not (p=0.3223).
- On the real IDEA-FAST wearable dataset, LongAgent performs comparably to the strongest baseline (RMSE 0.9937 vs 0.9939), with all methods showing R² near 0.02.
Strengths
Clean, leakage-aware problem formalization
Section 2 casts predictor discovery as a budgeted search over candidates c=(S,w,f) with strictly outcome-relative windows (Eq. 1), excludes previously evaluated candidates, and defines a closed action space (Eq. 2) — a precise, reusable formulation.
Large, controlled synthetic benchmark
360 datasets spanning six outcome-generation settings (Table 1), three coefficient magnitudes (κ ∈ {1, 0.5, 0.25}) and two noise levels, ten seeds each, with subject-wise 80/20 splits and held-out test reporting of RMSE/MAE/R².
Honest reporting of the real-data result
Ablation of the history design with statistics
Weaknesses
Tiny effect sizes relative to variance
Real-data experiment cannot support even a tie claim robustly
Ablation validates only one of two added evidence sources, on 10% of the data
Missing natural baselines
Synthetic ground truth is aligned with the search space and no code release
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
LongAgent (arXiv 2609.15859, Imperial College London) targets a genuine pain point in longitudinal medical modelling: hand-crafting predictors from heterogeneous, irregularly sampled variables. The authors formulate predictor discovery as a budgeted search over candidates c = (S, w, f) — a variable subset (at most ten variables), an outcome-relative time window, and an aggregation function (mean/median/p95/p05) — and delegate the search to an LLM agent that reads four evidence streams (search status, candidate history with losses, a coverage summary of evaluated combinations, and transition evidence linking past moves to loss changes) and acts via add/drop/replace/update/stop in a two-stage prompt (Section 2). The claims are: (1) the formulation is a novel framing for longitudinal outcome prediction; (2) LongAgent beats three non-agent search policies on 360 synthetic datasets; (3) ablations validate the history design; (4) on the real IDEA-FAST dataset it is comparable to the best baseline. The abstract is honest about point 4 — a point in the paper's favour.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is LongAgent?
LongAgent is an LLM-based agent from Imperial College London that automatically searches for predictive features in longitudinal medical data. Each search candidate combines a subset of variables, a time window before the outcome, and an aggregation function (mean, median, or percentiles). The agent uses a memory of past searches — including prior candidates, their prediction errors, which combinations were already tried, and how each move changed the loss — to decide its next step, and an elastic net model scores each candidate via cross-validated RMSE.
Does LongAgent outperform baselines on real clinical data?
No, not convincingly. On the real IDEA-FAST wearable dataset (1,084 participants predicting fatigue z-scores), LongAgent is essentially tied with the strongest baseline, rule-based search (RMSE 0.9937 vs 0.9939, R² ≈ 0.02). On 360 synthetic datasets it does achieve the best average RMSE (1.7376 vs 1.7527), but the improvement is statistically marginal (p=0.0273) and under 1% in relative terms.
Which large language models does LongAgent use?
Experiments on synthetic data use DeepSeek-V4-Flash, while the real-data experiments on IDEA-FAST use gpt-oss-120b, reportedly for privacy reasons. Both run at temperature 0.01 with thinking disabled, and the agent makes two-stage queries: first choosing a base candidate and action type, then choosing the action's parameters.
Is code available for LongAgent?
The paper text contains no code or data availability statement and does not include prompt templates, so exact replication from the paper alone is not possible. The synthetic data generation (equations and coefficient values), the search spaces, and the elastic net evaluation protocol are described in enough detail to support a reimplementation.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1 is only available through its caption; visual details of the agent input layout could not be inspected.
- No appendix, prompt templates, or supplementary material appears in the extracted text, so agent prompt design is assessed only from the Section 2 description.
- The absence of a code/data availability statement is based on the extracted text; a statement may exist in supplementary material.
- Statistical judgments rely on the values reported in Tables 2-4; raw per-dataset results were not available.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.