Coupled Scaling: A Representational Accessibility Framework for Neural Scaling Laws

Summary

The paper asks when two learning systems trained on the same task under a common resource protocol should share a scaling rate and when their rates should differ. Its answer is Coupled Scaling, a framework built on 'representational accessibility': support (what the architecture can ultimately represent), realized geometry, and acquisition (the order in which task-relevant directions are learned). In an orthogonal model with exact coefficients, unsupported target energy fixes the asymptotic floor while unacquired supported energy bounds the finite-budget residual. The central result (Proposition 2) shows that when acquisition can skip high-value directions no unique exponent exists: for target powers a_j ∝ j^(−b) (b>1) and prefix log-growth rate ρ, attainable exponents fill the sharp interval [ρ(b−1), min{b−1, ρb}], each realized by an explicit fixed order; a rank-window condition (Lemma 1) selects the product law α=ρ(b−1), and Corollary 1 transfers cross-task acquisition-rate reversals to exponent reversals. A fixed-kernel specialization recovers the classical spectral-tail rate via Karamata's theorem. Empirically, the paper reanalyzes released emergence trajectories of four open-weight models, finding that apparent implicit-curriculum orderings weaken under composite-matched contrasts (max-slope turns negative, AUC stays small and positive), and proposes—but does not run—a staged test where independently measured geometry predicts held-out loss. Key limitations: the theory's predictions are never directly tested, and the reanalysis is explicitly descriptive with uncalibrated permutation references.