A Systematic Evaluation of Machine Learning Methods for Fault Detection and Line Identification in Electrical Power Grids
Julian Oelhaf, Julian Oelhaf, Georg Kordowich, Georg Kordowich, Paula Andrea Pérez-Toro, Tomás Arias-Vergara, Andreas Maier, Andreas Maier, et al.
cs.LG · eess.SP
Quick verdict
Claims ML detects grid faults in 10ms windows (F1 0.991); systematic but single-topology PowerFactory study with no DL baselines and possible CV leakage.
Summary
This ICASSP 2025 paper benchmarks 14 classical machine-learning classifiers on two grid-protection tasks: binary fault detection (three-phase short circuits) and four-class faulty-line identification, using instantaneous voltage/current measurements from protection-relay devices. Data come from DIgSILENT PowerFactory electromagnetic-transient simulations of a 'Double Line' topology (one external grid, two loads, three buses, four transmission lines), with grid and fault parameters domain-randomized within physically constrained ranges (Table I). Episodes are trimmed ±80 ms around fault onset and sliced into overlapping 10–50 ms windows (9,600–48,000 features per window); a window is positive only when the fault onset lies fully inside it. Under 10-fold cross-validation (9:1 splits), 11 of 14 models exceed F1 0.96 for detection regardless of window length; linear models (logistic regression, Ridge, SGD) fall below 0.69, and AdaBoost also fails line identification (≤0.58). The best model reaches F1 0.991±0.018 with 0.342±0.509 ms per-window inference on an i7-13700K, while KNN is unusable at 88.6 ms. MLP, gradient boosting, and stacking best balance accuracy and runtime. Main limitations: a single small test topology, three-phase faults only (5% of real occurrences), no deep-learning baselines despite a DL-focused related-work section, no hyperparameter tuning protocol, F1-only reporting under heavy class imbalance (500/15,500 positive windows at 10 ms), and no code or data release; the random CV split risks episode-level leakage.
Review complete
Assessment at a glance
6
risk flags identified
● high 1
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- ML models can reliably detect three-phase short circuits and identify the faulty transmission line from instantaneous voltage/current measurements within a 10 ms window — half a 50 Hz period.
- The best-performing configuration achieves F1 0.991 ± 0.018 with 0.342 ± 0.509 ms per-window processing on a commodity Intel i7-13700K.
- This is the first evaluation of ML-based grid fault detection at the 10 ms lower bound of the real-time condition.
- Most evaluated models (11/14 for detection, 6/14 for line identification) perform well regardless of window length (10-50 ms), while linear models and AdaBoost fail.
- Domain randomization of grid parameters in PowerFactory simulations enables robust training on purely synthetic data, easing later transfer to real measurements.
Strengths
Genuinely unified comparison protocol
Fourteen classifiers are trained and evaluated on identical windows, features, splits and metrics (Section II-C, Figs. 2-3), directly addressing the incomparability of prior work the authors identify in the introduction; per-window-length results span 10-50 ms in a single framework.
Real-time framing backed by measured runtime
The window sweep is anchored to VDEW limits (25-40 ms by voltage level) and the IEEE 10-50 ms relay guidance [10],[12], and inference time is measured over 5,000 iterations per model (Fig. 4), exposing the decisive tradeoff: MLP 0.34 ms vs. KNN 88.6 ms against a 10 ms budget.
Physically grounded synthetic data generation
Honest scope statement
Weaknesses
Single tiny topology, generalization untested
Possible episode-level leakage in the CV split
No deep-learning or conventional-protection baselines
Thin statistical reporting under severe class imbalance
Underspecified model configuration and ambiguous labeling
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This ICASSP 2025 paper (arXiv:2609.16744) benchmarks 14 classical machine-learning classifiers on two power-grid protection tasks across sliding analysis windows of 10-50 ms: binary fault detection of three-phase short circuits, and four-class identification of the faulted line (Line12a/12b/23a/23b). The stated novelty is the first evaluation at the 10 ms lower bound of the real-time condition, motivated by VDEW detection limits (25-40 ms by voltage level) and the IEEE guide's 10-50 ms relay window [10],[12]. Headline results: F1 0.991 ± 0.018 at 0.342 ± 0.509 ms per-window processing. Fault localization is explicitly excluded. Claims are scoped to three-phase faults on one test topology, and the conclusion concedes the single-grid limitation — the framing is honest.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the best machine learning model for power grid fault detection in this paper?
The authors report that MLP, histogram-based gradient boosting and stacking performed best overall, balancing accuracy and speed. The single best configuration reached an F1 of 0.991 (±0.018) with about 0.342 ms of processing per 10 ms measurement window on an Intel i7-13700K — fast enough for real-time protection relays. KNN was accurate but far too slow at 88.6 ms, and simple linear models failed at both tasks.
How fast must a protection relay detect a fault?
The paper cites VDEW requirements of at most 25 ms detection time for extra-high voltage networks, 30 ms for high voltage and 40 ms for medium voltage, and an IEEE guide stating relays typically must detect faults within 10-50 ms. This is why the study tests analysis windows from 10 ms to 50 ms of instantaneous voltage and current data.
What data was used to train and test the fault detection models?
No real grid measurements: the dataset was generated with DIgSILENT PowerFactory electromagnetic transient simulations of a 'Double Line' test topology (four transmission lines, two loads, one external grid, eight relay devices). Grid and fault parameters — line lengths, loads, fault timing, fault resistance and more — were randomized within physically constrained ranges so models would be robust and could later transfer from simulation to real grid data.
Is the code or dataset from this paper publicly available?
The paper contains no code or data availability statement. The training data require the proprietary DIgSILENT PowerFactory simulator and an expert-built grid model, while the classifiers are standard scikit-learn implementations; exact replication would therefore require rebuilding the simulation setup from the parameter ranges given in the paper's Table I.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-4 (topology, F1 heatmaps, runtime plot) cannot be analyzed from extracted text; per-model/per-window F1 values are known only from prose aggregates in Sections III-IV.
- Hyperparameter details (SVM kernel, MLP architecture, tuning protocol) are absent from the extracted text, so the 'systematic' comparison may rest on default settings.
- arXiv v1 is dated 15 Sep 2026 while the embedded IEEE notice says (c) 2025 ICASSP — this review reflects the arXiv version as extracted.
- No code/data availability statement was found in the extracted text; the absence is inferred from this extraction, not an exhaustive check of supplementary material.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.