When Does Online Adaptation Pay on the Edge? A Leakage-Free Evaluation of Warmup, Learning-Rate Selection, and Resource Trade-offs for Time-Series Forecasting

Summary

This empirical evaluation-methodology study asks when online adaptation of time-series forecasting models pays off on edge devices, and shows the answer depends heavily on evaluation choices. Using six public multivariate streams (four ETT subsets, UCI Appliances, BDG2 smart-meter data), two backbones (DLinear and a compact PatchTST), and a leakage-free streaming protocol verified against a delayed-adaptation control, the authors identify two bias sources beyond known target leakage. First, the static baseline's warmup budget has a two-sided, non-monotone effect: adaptation-benefit estimates shift by 3.0-18.8 percentage points across a 1,000-20,000-step warmup range in all six dataset-backbone panels. Second, comparing SGD+m and Adam at a shared 1e-3 learning rate conflates optimizer quality with rate sensitivity; at that default Adam falls below the static baseline in 174/360 cells, but after per-optimizer rate rehearsal on a held-out pre-drift validation slice, Adam wins 310/360 cells with only 4 below static. The paper characterizes accuracy versus adaptation-state memory and A100-measured latency, finding parameter-efficient (head/calibration) variants nondominated on the memory axis, and shows BDG2 meter-selection rules swing reported gains (+13.5% vs +42.8% on PatchTST). Limitations include A100-only timing, a 5 W proxy for energy, only two backbones, descriptive seed standard deviations without significance testing, and a simplified calibration variant not compared to official PETSA.