Horizon-Aware Early Event Prediction for Tokamak Disruption Alarms
Takeshi Koshizuka, Takaharu Yaguchi, Takaharu Yaguchi
physics.plasm-ph · cs.LG
Quick verdict
Claims TLS best for finite-horizon disruption alarms on DIII-D/EAST; C-Mod fails; plausible benchmark but 3-seed evidence and calibration gap limit it.
Summary
This paper imports early event prediction (EEP) objectives from survival analysis into tokamak disruption alarm prediction. Using a shared causal encoder (GRU history branch plus an MLP over the last 50 ms), the authors compare Deep Survival Machines (DSM), a full-distribution baseline, with Temporal Label Smoothing (TLS), which predicts disruption probability within a fixed horizon, and survTLS, which models horizon-truncated within-horizon hazards, on public DisruptionBench data from DIII-D, Alcator C-Mod, and EAST. Evaluation combines threshold-free Deadline AUPRC with alarm metrics (Deadline Recall, Precision, F1, FAR/24h) under a validation-selected fixed policy, with a test-label oracle threshold reported as a diagnostic. TLS achieves the best mean alarm performance on DIII-D (Deadline AUPRC 0.959±0.005; F1 0.463±0.263) and EAST (0.968±0.014; F1 0.679±0.067); on C-Mod all methods fail (F1 ≈ 0.13–0.17). survTLS does not consistently beat DSM. The gap between validation-calibrated and oracle thresholds (TLS on DIII-D: F1 0.463 vs 0.805) shows threshold calibration is the main bottleneck. Optimal horizons and encoders vary by device; notably, a GRU-only encoder gives the paper's best result on EAST (F1 0.891). Main limitations: uncertainty rests on only three training seeds with large variances and no significance testing.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 2
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Adapting early event prediction objectives (TLS, survTLS) to tokamak disruption prediction aligns training with the finite operational warning horizon that fixed policies actually use.
- TLS achieves the best mean validation-selected alarm performance at the w=40 ms deadline on DIII-D and EAST, while survTLS does not consistently outperform DSM, suggesting within-horizon event-time modeling adds little value here.
- All methods perform poorly on Alcator C-Mod, and the results do not support a reliable method ranking on that device.
- The gap between validation-calibrated and test-label oracle thresholds (e.g., TLS on DIII-D: F1 0.463 vs 0.805) shows that threshold calibration on validation data is a major bottleneck.
- Selected horizons and encoder architectures vary across devices, reflecting device-specific disruption characteristics.
Strengths
Operational-alignment framing is correct and well-motivated
Section 1 and §2.2 argue that alarms depend on finite-horizon risk rather than the full residual-time distribution, and the DSM/survTLS/TLS taxonomy (beyond-horizon fitting vs within-horizon timing) is clearly laid out before any experiments.
Evaluation protocol separates deployable from diagnostic evidence
§4.5 splits validation-selected fixed-policy results (Table 2, threshold calibrated at ~1% normal-shot FPR on a calibration split) from a clearly labeled test-label oracle FAR-constrained diagnostic (Table 3), with alarm event accounting (Appendix C) defined precisely; test data never enters checkpoint, configuration, or threshold selection.
Honest reporting of failure and calibration gaps
Controlled comparison design
Weaknesses
Statistical support is thin
Horizon selection asymmetry between methods
Best result contradicts the headline architecture
C-Mod failure mode unexplored
No code release and unspecified hyperparameter provenance
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper adapts early event prediction (EEP) objectives — Temporal Label Smoothing (TLS, Yèche et al. 2023) and horizon-truncated survTLS (Yèche et al. 2024) — to survival-based tokamak disruption alarm prediction, taking Deep Survival Machines (DSM, Nagpal et al. 2021) as the full-distribution baseline. On public DisruptionBench data (DIII-D, Alcator C-Mod, EAST; Table 1) it compares the three objectives under a shared causal encoder (GRU history branch + 50 ms recent-window MLP), a common 5 ms causal resampling, and an EEP-style evaluation with a w=40 ms deadline. Three claims are stated and mostly match the results: (1) TLS gives the best mean validation-selected alarm performance on DIII-D and EAST; (2) survTLS does not consistently beat DSM; (3) all methods fail on C-Mod, where no reliable ranking is possible.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does this paper do in tokamak disruption prediction?
It adapts two 'early event prediction' training objectives from survival analysis — Temporal Label Smoothing (TLS) and horizon-truncated survTLS — to tokamak disruption alarms, and compares them against Deep Survival Machines (DSM) on the same neural encoder using public DisruptionBench data from DIII-D, Alcator C-Mod, and EAST.
Which method predicts disruptions best according to the paper?
TLS gives the best mean alarm performance on DIII-D and EAST (e.g., Deadline F1 0.679±0.067 on EAST), while on Alcator C-Mod all methods perform poorly (F1 around 0.13–0.17) and no reliable ranking emerges. Notably, the paper's single best result (EAST, F1 0.891) uses TLS with a GRU-only encoder rather than the full fused encoder.
What is Early Event Prediction (EEP) and how does it differ from standard survival analysis?
Standard survival analysis models the complete distribution of time to an event, including events far in the future. EEP instead trains the model only on whether the event occurs within a finite, operationally relevant horizon — here 20–120 ms — so the training objective matches the alarm decision actually deployed.
Is the code or data available?
The paper uses the publicly released DisruptionBench databases for DIII-D, Alcator C-Mod, and EAST. No code release is mentioned in the extracted text; the described implementations build on Auton-Survival (DSM) and the official TLS/survTLS reference code from prior publications.
What is the biggest limitation of the study?
Statistical support: all uncertainty comes from just three training seeds on one data split, with large variances and no significance tests; validation-based threshold calibration also leaves large headroom (on DIII-D, TLS F1 rises from 0.463 with the deployable threshold to 0.805 with a test-label oracle threshold).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures were not part of the extracted text; all assessment relies on tables and equations rendered inline, which appear complete.
- Appendix B.3 (optimization details referenced in the text) is effectively empty in the extraction, so some reproducibility detail is unverifiable.
- Single-seed values (Table 4 horizon sweep; low/high FAR budget sweeps) cannot be cross-checked for variance from the text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.