Reliability-Centered Evaluation of Sparse Longitudinal CT Lesion-Size Forecasting with Conformal Interval Calibration and Gompertz-Inspired Regularization
cs.LG · q-bio.QM · stat.ME
Quick verdict
Constructs a 205-trajectory DeepLesion-DLT CT forecasting benchmark; finds only the most recent extra visit helps; rigorous but small test set limits power.
Summary
This paper builds a five-visit same-lesion longitudinal benchmark from DeepLesion and Deep Lesion Tracker (205 trajectories, 129 patients) to evaluate sparse CT lesion-size forecasting under a reliability-centered lens. Using a fixed visit-index horizon design that always predicts the T3→T4 log-volume change, the author compares an MLP, a cohort-level feature Gaussian process, MC Dropout, a 5-member deep ensemble, and a Gaussian residual-scale baseline, plus patient-level conformal calibration and a Gompertz-inspired regularizer. Mean held-out RMSE across ten seeds was 0.4726, 0.4305, 0.4499, and 0.4513 for m=1–4: one extra recent observation helped, older history did not, and simple baselines (linear extrapolation RMSE 0.4191 at m=4) remained competitive. Raw 95% coverage was near nominal only for the GP; MC Dropout and ensembles were conservative, and conformal calibration reached nominal coverage only with wider intervals. Development cross-validation selected no Gompertz regularization (λ*=0), whose population reference opposed lesion-level change directions in over half of cases, and the one-time held-out λ=0.1 contrast showed no uniform benefit. Main limitations: 38 test trajectories from 26 patients, a 19-patient calibration set that makes 95% conformal calibration degenerate, visit ordering derived from scan identifiers rather than acquisition dates, and a RECIST-based ellipsoid volume proxy.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Under a fixed visit-index horizon (always predicting the T3→T4 log-volume change), one additional recent observation improves RMSE (0.4726→0.4305), but progressively older history (m=3,4) adds no further benefit.
- Methods with similar point accuracy show distinct uncertainty behavior; patient-level conformal calibration achieves near-nominal or conservative coverage only at the cost of wider intervals.
- Patient-grouped development cross-validation selects no Gompertz-inspired regularization (λ*=0), and the population reference opposes lesion-level change directions in over half of cases; the held-out λ=0.1 contrast shows no uniform accuracy or reliability benefit.
- Conventional longitudinal baselines (linear extrapolation, ridge, LOCF) remain competitive with the neural methods on this benchmark.
- Predictive accuracy, uncertainty reliability, and trajectory consistency capture distinct aspects of performance and should be evaluated jointly in sparse longitudinal imaging.
Strengths
Fixed-horizon design removes a classic confound
The primary design (§2.6, §4.2) pins the target Δ3→4 and latest observation T3 while varying history m∈{1..4}, isolating the value of older observations; the conventional sparse-to-final analysis is explicitly demoted to descriptive, which many history-depth studies fail to do.
Leakage-aware, disciplined pipeline
Patient-level 140/27/38 split (Table 2), calibration patients used only for conformal thresholds, λ selected by development-CV before one-time test access (§3.6–3.7), and deterministic tie-breaking that uses no future measurements (§2.2–2.3); the 7,776-record directionality audit (§4.1) is unusually thorough.
Mixture-consistent uncertainty evaluation
Transparent negative results
Strong reproducibility posture
Weaknesses
Statistical power cannot support the headline reliability comparisons
No inferential support for the central history-depth claim
Temporal foundation is operational, not clinical
Benchmark value for deep models is undercut by baselines
Target noise floor never quantified
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This single-author paper (arXiv 2609.21197, Vanderbilt mathematics department) builds a five-visit same-lesion CT benchmark from DeepLesion RECIST measurements linked by Deep Lesion Tracker (DLT) matches — 205 trajectories from 129 patients after patient/scan-level QC — and evaluates sparse longitudinal lesion-size forecasting jointly on point accuracy, uncertainty reliability, post-hoc conformal calibration, subgroup behavior, and a Gompertz-inspired trajectory regularizer. Three central claims: (1) under a fixed visit-index horizon (always predicting the T3→T4 log change), only one additional recent observation helps and older history adds nothing; (2) similar RMSE conceals very different uncertainty behavior, and conformal calibration buys coverage with width; (3) development-CV selects no Gompertz regularization (λ*=0) and the population reference frequently opposes lesion-level change. The claims are deliberately modest and consistently matched by the evidence; this is a benchmark-and-diagnostics paper, explicitly not a clinical validation (§6–7).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the DeepLesion-DLT longitudinal benchmark introduced in this paper?
The paper assembles 205 same-lesion trajectories from 129 patients, each with five ordered CT visits, by linking DeepLesion RECIST measurements through Deep Lesion Tracker (DLT) longitudinal matches, followed by patient- and scan-level quality control. It is used to study predicting lesion-size change from sparse follow-up imaging.
Does adding more historical CT scans improve lesion-size prediction according to this study?
Only partially. Predicting the T3→T4 log-volume change, mean held-out RMSE improved from 0.4726 with one observation (m=1) to 0.4305 with two (m=2), then stayed flat at m=3 (0.4499) and m=4 (0.4513). Older history added no further benefit, and simple baselines such as linear extrapolation (RMSE 0.4191 at m=4) remained competitive with neural models.
Did the Gompertz-inspired regularization help in this paper?
No. Patient-grouped cross-validation on the development set selected λ*=0, and a one-time held-out comparison of the best nonzero weight (λ=0.1) against unregularized MC Dropout showed no consistent accuracy or reliability benefit. The population Gompertz reference disagreed with the direction of actual lesion change in more than half of cases.
Is the code and data for this paper publicly available?
Yes. The study uses the public DeepLesion and Deep Lesion Tracker datasets, and the author states that processing scripts, benchmark-construction code, and analysis code are available at github.com/Lorettakong/deeplesion-dlt-reliability, with instructions to reconstruct the benchmark from the original public releases.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures are available only as extracted text descriptions; reliability diagrams (Fig. 11A), bootstrap CI widths in Fig. 8, and curve shapes in Figs. 6/7/9/10 could not be visually inspected.
- Some per-experiment seed standard deviations (e.g., Experiment 1) appear only in figures, so seed-level variability for the headline RMSE comparison could not be verified numerically.
- Review is based on arXiv v1 (18 Sep 2026), a preprint in Elsevier template not yet peer reviewed.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.