GraphToolbox: A Configurable Python Framework for Graph Neural Network Forecasting
Eloi Campagne, Yvenn Amara-Ouali, Yannig Goude, Argyris Kalogeratos
cs.LG · cs.AI
Quick verdict
GraphToolbox unifies GNN load forecasting (51 PyG convs); French-load gains are real but modest, classical baselines win on net-load. Honest, rigorous software.
Summary
GraphToolbox is an open-source Python framework built on PyTorch Geometric that unifies graph construction, model selection, online aggregation, interpretability, and significance testing behind a configuration-driven pipeline for electricity load forecasting. Its adapter runs 51 of 65 PyG convolutions end-to-end, plus recurrent cells from PyTorch Geometric Temporal. In the French regional load case study (2019 test year), 48 convolutions span 1.14%–1.60% MAPE and online expert aggregation (MLpol) improves this to 0.98% at 750 MW RMSE, beating additive models, gradient boosting, an LSTM, and Chronos foundation models. An identity-graph ablation, however, attributes only a median 23 MW to message passing itself. On net-load, direct graph models trail classical additive models, and per-component forecasting (load, wind, solar) narrows but does not close the gap. Interpretability diagnostics reveal nearly uniform, unstable attention edge rankings, and a seed study shows single-run entries may be favorable draws since seed variability can exceed headline margins. Validation rests on a single 12-node case and one test year, and scalability is not demonstrated beyond 500 nodes. Code, data, and cached forecasts are public, and the significance tooling (Diebold–Mariano, Holm correction, moving-block bootstrap, Model Confidence Set) is unusually rigorous.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A signature-reading ConvAdapter lifts 51 of 65 PyTorch Geometric convolutions (78.5%) into one forecasting meta-model, enabling catalog-wide sweeps from a single configuration-driven code path.
- On French regional load (2019 test year), online MLpol aggregation over the convolution sweep reaches 0.98% MAPE / 750 MW RMSE, beating additive models, gradient boosting, an LSTM and Chronos foundation models.
- The identity-graph ablation shows message passing itself contributes only a median 23 MW of that gain, so most of the improvement over classical baselines comes from architecture and preprocessing.
- On net-load, direct GNNs underperform classical additive models, and per-component forecasting (load/wind/solar) improves 9 of 10 convolutions by ~162 MW on average without closing the gap.
- Attention-based edge attributions are nearly uniform and unstable on this graph, so explanation maps should be treated as diagnostics, not evidence of spatial structure.
Strengths
Full-pipeline unification with genuinely broad operator coverage
Section IV documents an end-to-end coverage benchmark: 51 of 65 torch_geometric.nn.conv operators (78.5%) run inside the myGNN meta-model without modification; the ConvAdapter (Section III.B, ~240 lines) routes constructor and forward signatures per operator, enabling catalog-wide sweeps like Table V from a single code path.
Statistical discipline unusual for a systems paper
Section III.E layers Diebold–Mariano tests with Newey–West variance and HLN small-sample correction, Holm–Bonferroni adjustment, moving-block bootstrap (one-day blocks, 2000 resamples), and the Model Confidence Set; Section V.A uses them to report that 36 of 48 operators are statistically indistinguishable at 90% rather than crowning a spurious winner.
Honest negative and null results
Explicit result provenance and reproducibility package
Practical scale guidance
Weaknesses
External validity confined to one small network and one test year
Mechanism of the headline gain is not relational
Reconciliation evaluated only with a weak top-level forecast
GNN–baseline comparisons are not configuration-matched
Seed study covers only leading models
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is a software/framework paper: GraphToolbox packages graph construction, architecture benchmarking, online aggregation, interpretability and forecast-comparison statistics into one configuration-driven pipeline on PyTorch Geometric (Section III, Figure 1). The headline claims are (1) the ConvAdapter runs 51 of 65 PyG convolutions end-to-end (78.5%, Table IV); (2) on French regional load, the 48-operator sweep spans 1.14–1.60% MAPE with MLpol aggregation at 0.98% / 750 MW RMSE, beating GAM, XGBoost, LSTM and Chronos baselines (Table V); (3) on net-load, direct GNNs lose to a classical GAM and component-wise forecasting narrows but does not close the gap (Table VI). The paper is careful to disclaim the mechanism: the identity-graph ablation attributes only a median 23 MW to message passing, and Table II shows the tabulated single-run entries sit 58–147 MW below 5-seed means.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is GraphToolbox?
GraphToolbox is an open-source Python framework (GPL-3.0, pip-installable) built on PyTorch Geometric for graph-based electricity load forecasting. It unifies graph construction, model selection across 51 of 65 PyG convolution operators, online forecast aggregation, interpretability tools, and statistical significance testing behind configuration files, so swapping the graph or architecture is a one-field change.
Does GraphToolbox beat classical forecasting baselines?
It depends on the target. On French regional gross load, its graph models and MLpol aggregation (0.98% MAPE, 750 MW RMSE) beat additive models, gradient boosting, an LSTM and Chronos. On net-load, direct graph models lose to classical additive models; forecasting load, wind and solar separately narrows but does not close that gap.
Are graph neural networks actually better for electricity load forecasting according to this paper?
The paper's own ablations suggest caution: an identity-graph comparison shows message passing contributes only a median 23 MW of the improvement on gross load, and on net-load the graph models trail a classical GAM. The gains largely come from the architecture, preprocessing and aggregation rather than the relational inductive bias itself.
Is the code and data available for reproduction?
Yes. The paper states the code, documentation and notebooks are on GitHub, installable via pip with a lock file, pinned seeds and stated hardware; cached forecasts allow rerunning all bootstrap and Diebold–Mariano tests without retraining, and trained weights ship as a release asset. Data are public RTE and Météo-France observations (2015–2019).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1 (pipeline diagram) is not analyzable in detail from the extracted text.
- The full 48-operator sweep table is not included in the extract; only 10 representative rows appear, so per-operator claims beyond the quoted range/median cannot be verified.
- Figure 2's maps are described only via their caption text; visual details of the explanation graphs cannot be assessed.
- Baseline provenance (four rows borrowed from arXiv:2507.03690) means those comparisons inherit that study's configurations.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.