Online Gradient Computation for Warping Gaussian Process Transformations

Summary

The paper targets online inference for warped Gaussian processes, where a monotonic parametric transformation g(y;θ) maps non-Gaussian observations to latent targets modeled by a standard GP. Existing streaming variants either re-optimize warping parameters only periodically (Kou et al., 2013) or trade analytical tractability for higher model capacity (Bui et al., 2016). The core contribution is Theorem 3.1: the gradient of the instantaneous negative log-likelihood admits an exact recursive evaluation, requiring a single extra state variable Bn beyond the standard recursive GP states (αn, Ωn); the proof in supplement S1 exploits the rank-one recursive structure of Ωn. Algorithm 1 builds on this to jointly update latent GP moments and warping parameters via projected Adam, and adds approximate-linear-dependency (ALD) sparsification for O(Dn²+rDn) per-step cost. On a 1D synthetic benchmark (a cubic-root-warped noisy sinusoid; 1000 streams of 101 points), the dictionary saturates at 68 atoms and the method reaches −0.28 nats per observation, 0.76 nats better than a standard GP under an otherwise identical configuration; the learned warping closely matches the true cubic transform. Main limitations: validation rests on one small synthetic task with no comparison against the closest streaming competitors; kernel and noise hyperparameters are fixed (σ matches the true noise); the joint loop lacks convergence analysis, since its state variables embed targets warped at stale θ values; and no dispersion across the 1000 runs is reported.