Emergence of Fibrations, Compression, and Symmetry Breaking in Artificial Neural Networks

Summary

The paper argues that deep-network training spontaneously generates local graph symmetries — fibrations (isomorphic input trees), opfibrations, and their combination, coverings — detectable in trained weights via a balanced-coloring algorithm. Theoretically, it proves that fibers synchronize forward activity and opfibers synchronize backpropagated errors, that weight updates become identical within covers ('synchronized learning'), and — via a Cover Coarse-Graining Theorem — that covering partitions only coarsen under gradient descent, from which, citing prior work, covers are stable SGD attractors. Merging cover nodes with a sum/mean rule preserves forward computation exactly for exact symmetries; with thresholds, quasi-covers yield approximate compression bounded by K·εop·εfib. Empirically, a three-layer MNIST MLP compresses to 17% of its size with no loss or accuracy change; an ImageNet CNN and a Multi30k Transformer compress to 18–70% depending on loss tolerance; covers in a PPO-trained LSTM emerge and then break under entropy-driven exploration. The authors attribute node collapse and loss of plasticity to cover-induced parameter tying, and propose Fibration Symmetry Breaking (compress to the covering base, then add random naive nodes), which outperforms Continual Backpropagation on sequential ImageNet (95% vs 90% at task 5,000). Limitations: small-scale benchmarks, thin baselines (random/L2 pruning; DCL only), approximate error bounds, and no visible variance or code availability.