Loss Choice or Model Choice? The Role of Forecast Level in Cryptocurrency Volatility Forecasting
Andrzej Tokajuk, Jarosław A. Chudziak, Jarosław A. Chudziak
q-fin.CP · cs.AI · cs.CE · q-fin.RM
Quick verdict
Claims raw loss effects in crypto volatility forecasts are mostly forecast level; after alignment model choice dominates, shared-input check inconclusive.
Summary
This paper asks whether the training loss or the forecasting model matters more for cryptocurrency volatility forecasts, and whether loss effects are mainly differences in forecast level rather than day-to-day dynamics. The authors run a walk-forward comparison of seven losses (MSE-log, Huber-log, Pinball, QLIKE, Patton-b, HMAE, HMSE) and five models (HAR, HAR-X, DLinear, NeuralHAR, LightGBM) on daily Binance data for BTC, ETH, BNB, XRP and ADA, targeting next-day Garman-Klass variance across 25 asset-fold blocks. A ratio of average absolute marginal QLIKE differences shows losses vary about 2.9 times more than models (median 2.91, 95% CI [2.27, 3.25]) before adjustment. A validation-based constant rescaling (alignment) explains 56-89% of cross-loss forecast differences (LevelShare), cuts cross-loss variation by 77% overall, and flips the ratio to 0.67, favouring model choice in the full grid — though the shared-predictor check (0.81, CI [0.66, 1.22]) is inconclusive. For one-day 5% Gaussian VaR, raw breach rates span 1.3-6.3% across losses but narrow to 3.5-3.6% after alignment, with Kupiec rejections falling from 128 to 109 of 175 series. Limitations: five assets and a one-day horizon, QLIKE-based alignment partially matching the primary evaluation score, a possibly HMSE-driven raw loss effect, and no released code.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 19 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Before level adjustment, marginal QLIKE variation across the seven training losses is about 2.9 times larger than across the five models (median ratio 2.91, 95% CI [2.27, 3.25]).
- A validation-based constant level alignment explains 56–89% of cross-loss forecast differences (LevelShare) and reduces cross-loss QLIKE variation by 77% overall (95% CI [72%, 79%]).
- After alignment, the loss-to-model ratio falls to 0.67 (95% CI [0.56, 0.99]), making model choice the larger source of variation in the full grid, though the shared-predictor estimate (0.81, CI [0.66, 1.22]) is inconclusive.
- Cross-loss differences in one-day 5% VaR breach rates (raw range 1.3%–6.3%) narrow to 3.5%–3.6% after alignment, indicating raw risk differences stem mainly from persistent forecast-level shifts rather than daily dynamics.
Strengths
Well-posed estimand question grounded in point-forecast theory
Table 1 explicitly maps each of the seven losses to its conditional forecast target (MSE-log → geometric mean, QLIKE/Patton-b → mean, HMAE → h-weighted median), building the design on Gneiting (2011) and Patton (2011) rather than treating loss as a mere tuning knob.
Leakage-aware, paired evaluation design
The alignment constant is estimated on validation data only (Sec. 3.2, Eq. 1); LevelShare is computed on test forecasts independently of the alignment constants; stationary bootstrap resamples identical dates across all forecasts, so comparisons are properly paired (Sec. 4).
Multi-axis robustness of the headline raw ratio
Downstream risk translation with coverage testing
Honest scoping of the central reversal
Weaknesses
Model factor conflates architecture with inputs and history length
Raw loss effect may be an HMSE artifact
Partial circularity between alignment criterion and evaluation score
Persistent VaR miscalibration underplayed by the narrowing headline
No code or pipeline release; incomplete model specification
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks a deceptively simple attribution question: when volatility-forecasting benchmarks report large differences between training losses, is the loss really changing forecast behaviour, or is it only shifting the overall forecast level? Building on the point-forecast functional literature (Gneiting 2011; Patton 2011; Hansen–Lunde 2006), Table 1 makes each loss's implicit estimand explicit — MSE-log targets the conditional geometric mean, QLIKE/Patton-b the conditional mean, HMAE an h-weighted median — so different losses legitimately produce different forecast levels. Three research questions structure the work: the size of loss vs model effects (RQ1), the level-vs-dynamics decomposition (RQ2), and consequences for one-day VaR (RQ3). The central claims are quantitative: the raw loss effect is ~2.9× the model effect; level alignment explains most of it; model choice dominates after alignment; and cross-loss VaR breach differences collapse after alignment.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the main finding of this paper?
In cryptocurrency volatility forecasting, the large raw performance differences between training losses mostly reflect persistent differences in forecast level, not different daily dynamics. After a validation-based constant level alignment, model choice becomes the larger source of score variation (ratio 0.67 vs 2.91 before), and cross-loss VaR breach rates collapse from a 1.3%–6.3% range to 3.5%–3.6%.
Which loss function is best for volatility forecasting?
None universally. The paper shows loss rankings depend on the evaluation score (Spearman correlation −0.200 between rankings under QLIKE and MSE-log). QLIKE and Patton losses evaluate best under QLIKE, but because each loss targets a different conditional quantity (mean, median, geometric mean), 'best' is score-relative rather than absolute.
Does the training loss matter for Value-at-Risk?
Raw one-day 5% Gaussian VaR breach rates vary from 1.28% (HMSE) to 6.31% (Pinball) across seven losses. After a constant level alignment calibrated on validation data, all losses give breach rates of 3.5%–3.6%, so most raw risk differences come from forecast level. However, all aligned rates remain below the nominal 5%, and the Kupiec test still rejects coverage for 109 of 175 series, so alignment does not produce correctly calibrated VaR.
Is code or data available for this paper?
The paper states no code or data release. The inputs are public (Binance daily OHLCV through 31 May 2026) and the models are standard (HAR, HAR-X, DLinear, LightGBM, a HAR-style neural model), so reimplementation is feasible at CPU scale, but exact replication is limited because the NeuralHAR architecture and LightGBM hyperparameters are not specified in the text.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Review based solely on the extracted text; no figures were present in the extraction (tables were embedded and analysable).
- NeuralHAR architecture and LightGBM hyperparameters may be documented in an appendix not included in the extracted text.
- v1 preprint dated 22 Sep 2026; not yet peer reviewed at time of writing.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.