Learning from VAE Errors to support ECG-based Differential Diagnosis of Myocardial Scar
Shayan Sharifi, Riccardo Treu, Ilaria Gandin, Federico Garoia, Marco Merlo, Giulia Cisotto
cs.LG · stat.ML
Quick verdict
Claims VAE reconstruction errors flag myocardial scar from ECG (AUROC 0.643): honest but modest results; small cohort, grid-picked configs, low specificity.
Summary
The paper tests whether ECG-derived representations can distinguish cardiomyopathy patients with (LGE+, n=174) versus without (LGE-, n=126) myocardial scar, as defined by late gadolinium enhancement CMR, in a single-center cohort of 300 subjects. Three representations are compared: 32-dimensional latents from the pretrained ECGx.AI β-VAE foundation model (trained on ~1.1 million ECGs), latents from a shallower β-VAE trained by the authors on 3,000 normal PTB-XL ECGs, and a 12-dimensional vector of lead-wise DTW-based reconstruction errors from the shallow model. Five classifiers are evaluated under three cross-validation schemes with inner 3-fold tuning and leakage-aware normalization. Best results: ECGx.AI latents reach AUROC 0.686 (Random Forest, 5-fold), shallow latents only 0.577 (Gradient Boosting, LOSO), while DTW errors reach 0.643 (Logistic Regression, 80/20 split); Mann-Whitney U tests show significant class differences in 10 of 12 leads. Limitations include very low specificity in several headline configurations (0.191 for the best DTW pipeline), selection of top-performing grid configurations without a separate held-out confirmation, uncorrected multiple testing across leads, a confounded foundation-versus-shallow comparison (training scale and loss function differ), no external validation, and no code release.
Review complete
Assessment at a glance
6
risk flags identified
● high 1
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- ECGx.AI foundation latents classify LGE+/LGE- with AUROC 0.686 (Random Forest, 5-fold CV) in a 300-patient DCM/NDLVC cohort.
- DTW-based reconstruction errors from a shallow β-VAE trained only on normal PTB-XL ECGs discriminate scar with AUROC 0.643 (Logistic Regression, 80/20 split).
- Lead-wise DTW reconstruction errors differ significantly between LGE+ and LGE- patients in 10 of 12 leads (two-tailed Mann-Whitney U).
- A lightweight β-VAE pipeline with error-based features approaches foundation-model performance for this specific differential diagnosis, offering an interpretable screening alternative.
Strengths
Honest reporting of modest performance
Tables 1-2 report AUROC 0.577-0.686 with dispersion instead of inflated claims; Section 3 explicitly invokes the reconstruction-classification trade-off [12], and the Conclusions request external multi-center validation rather than asserting clinical readiness.
Leakage-aware, split-controlled protocol
Section 2: normalization is fitted strictly within each partition; identical data splits are reused across representations and models; hyperparameters are tuned by inner 3-fold grid search within each outer split; fixed random seeds are used.
Interpretable error-based representation
Multi-regime cross-validation
Weaknesses
Screening-incompatible specificity in headline configs
Best-of-grid selection without confirmation
Confounded foundation-vs-shallow comparison
Missing MSE-error ablation
OOD shift confounds the error signal
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This exploratory study asks whether routine 12-lead ECGs can triage cardiomyopathy patients (DCM/NDLVC) for CMR by predicting LGE-defined myocardial scar. The cohort is 300 single-center patients (174 LGE+, 126 LGE-) with expert-annotated CMR. Three representations are compared: (1) 32-d 'AI Factors' from the pretrained ECGx.AI β-VAE foundation model (trained on ~1.1M Utrecht ECGs); (2) 32-d latents from a shallower 4-layer β-VAE trained by the authors on 3,000 normal PTB-XL ECGs; (3) a 12-d vector of lead-wise DTW-based reconstruction errors from that shallow model. The central claim is twofold: foundation latents beat shallow latents (AUROC 0.686 vs 0.577 under matched GB/LOSO), and error-based features from the lightweight model approach foundation performance (0.643), making them an interpretable screening alternative.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What accuracy did the model achieve for detecting myocardial scar from ECG?
In this single-center study of 300 cardiomyopathy patients, the best AUROC was 0.686 using latent features from the pretrained ECGx.AI foundation model with a Random Forest classifier. Errors from a lightweight β-VAE reached 0.643 and the lightweight model's own latents only 0.577 — modest values, not clinically validated.
How does the method work without disease-labeled training data?
A β-variational autoencoder is trained only on normal ECGs (3,000 from PTB-XL). When it reconstructs patient ECGs, reconstruction error per lead — measured with Dynamic Time Warping — is larger where the ECG deviates from normal patterns, and these 12 error values are used as features for classifying scar-positive vs scar-negative patients.
Is the code or data from this paper available?
The paper does not mention a code or weights release, despite claiming fixed random seeds. The patient cohort is private (anonymized, consented), and the authors state that the ECGx.AI foundation model could not be fully reproduced because not all details were available. The PTB-XL training data is public.
Can this ECG method replace cardiac MRI for scar detection?
No. The authors explicitly position the tool as a screening aid to prioritize which patients should undergo cardiac MRI. Performance is modest (best AUROC 0.686), specificity is very low in several configurations (as low as 0.191), and external multi-center validation has not been performed.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-3 are available only via their captions in the extracted text; plotted statistics (which two leads are non-significant in Fig. 3, exact p-values, per-sample reconstruction curves) could not be verified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.