Muon-C: Operator-Aligned Muon for Convolutional Kernels
cs.LG · stat.ML
Quick verdict
Muon-C polarizes Fourier channel-transfer blocks for conv kernels; theory is clean and gains real but largest early in training and at small scale.
Summary
Muon-C modifies the Muon optimizer for convolutional kernels. Standard Muon unfolds a kernel into a Cout×(Cin·kh·kw) patch matrix, which the authors argue reflects a local patch map rather than the convolution operator. Muon-C instead applies an orthonormal kh×kw DFT to the kernel momentum, treats each frequency's Cout×Cin channel-transfer matrix as an independently polarized block, and inverts the transform, so updates map exactly back to the finite kernel support. Theoretically, the paper proves the update is the exact linear minimization oracle for the critically sampled convolution norm, and that its worst-case distortion relative to the continuous convolution-operator norm (Λ_kh·Λ_kw ≤ √n) is never worse than unfolding and strictly better for 3×3 kernels (25/9 vs 3). Empirically, on CIFAR-10 flow matching with matched applied-update RMS, Muon-C reaches 9.87 FID at 40k steps versus 22.26 (unfolded Muon) and 51.31 (Adam), and matches the baselines' final quality with 0.62×/0.64× model FLOPs (optimizer cost excluded; wall-clock savings shrink to 14–27%). Gains persist on ImageNet-1k-32 and ImageNet-100 classification (ResNet-34/50, ConvNeXtV2-T), though endpoint classification margins are 0.24–0.60 points. A matched spatial-block ablation (Muon-S) supports frequency organization as the driver. Limitations: no code release, small-scale evidence, and minor numerical inconsistencies between Tables 5 and 6.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A unitary change of basis (including Fourier) cannot alter the global Muon direction; only the partition into independently polarized blocks changes the optimizer geometry (Proposition 1, Sec. 4.3).
- Muon-C's update is the exact linear minimization oracle for the critically sampled convolution norm ‖U‖samp, with updates mapping exactly back to the finite kh×kw kernel support (Proposition 2, Eq. 2–4).
- Relative to the continuous convolution-operator norm, Muon-C's worst-case distortion (Λ_kh·Λ_kw ≤ √n) is never worse than unfolding's (√n) and strictly better for 3×3 kernels: 25/9 vs 3, i.e., oracle fractions 9/25 vs 1/3 (Theorem 3, Corollary 4).
- Under matched applied-update RMS on CIFAR-10 flow matching, Muon-C reaches 9.87 FID at 40k steps vs 22.26 (unfolded Muon) and 51.31 (Adam), and matches their final quality with 0.62×/0.64× of their model FLOPs.
- Frequency-wise organization drives the gain, not blockwise polarization alone: the matched spatial-block ablation Muon-S reaches only 22.28 FID at 40k vs 9.87 for Muon-C, converging to 3.49 vs 3.47 at 400k.
Strengths
Crisp conceptual separation of coordinates vs. blocks
Proposition 1 proves the polar map is unitarily equivariant, so a Fourier change of basis alone returns exactly unfolded Muon; the geometry change must come from the independent block partition (Sec. 4.3, Fig. 3's non-collinear two-coefficient example). This pre-empts a common misconception about 'Fourier Muon'.
Unified norm theory with explicit constants
Theorem 3 chains ‖U‖sp ≤ ‖U‖patch ≤ ‖U‖samp ≤ ‖U‖conv and bounds distortions at n / √n / Λ_kh·Λ_kw; for 3×3, Λ3 = 5/3 gives 25/9 vs 3 vs 9, and Corollary 4 converts this to continuous-oracle fractions 9/25 vs 1/3 vs 1/9 — a genuinely informative ranking, not just asymptotics.
Above-average experimental hygiene for an optimizer paper
Mechanistic diagnostics beyond end metrics
Honest cost accounting
Weaknesses
Headline gains are early-training; endpoint effects are small
Efficiency framing overstates practical savings
No code or data release
Cross-table numerical inconsistencies
Small-scale evidence and theory gaps for real architectures
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper targets a well-posed question: which matrix geometry should Muon act on for convolutional kernels? Standard practice unfolds W ∈ R^(Cout×Cin×kh×kw) into a Cout×(Cin·kh·kw) patch matrix. The authors claim (i) unfolding is one arbitrary choice, (ii) translation equivariance singles out frequency-wise Cout×Cin channel-transfer matrices A_W(θ) (Eq. 1) as the operator-aligned objects, and (iii) a critical kh×kw Fourier grid lets each block be polarized independently while mapping updates exactly back to the finite kernel support. Central results: Proposition 1 (a unitary Fourier change of basis alone returns unfolded Muon exactly — geometry lives in the block partition, not coordinates); Proposition 2 (Muon-C is the exact linear minimization oracle for the critically sampled convolution norm, Eq. 2); Theorem 3 (‖U‖sp ≤ ‖U‖patch ≤ ‖U‖samp ≤ ‖U‖conv, with distortions 9 / 3 / 25/9 for 3×3 kernels via Λ3 = 5/3); Corollary 4 (continuous-oracle fractions 1/9, 1/3, 9/25). Empirical claims center on CIFAR-10 flow matching (9.87 vs 22.26 vs 51.31 FID at 40k under matched update RMS) with transfer to ImageNet-1k-32 and ImageNet-100 classification.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Muon-C?
Muon-C is an extension of the Muon optimizer for convolutional kernels. Instead of unfolding a kernel into one large Cout×(Cin·kh·kw) matrix, it transforms the kernel momentum with a kh×kw critical Fourier grid, polarizes each frequency's Cout×Cin channel-transfer matrix independently, and transforms back — so updates stay exactly on the original kernel support and align with the convolution operator's translation-equivariant structure.
How much faster is Muon-C than unfolded Muon?
On CIFAR-10 flow matching it reaches unfolded Muon's final 400k-step quality using about 0.62× the model FLOPs (optimizer cost excluded; measured wall-clock saving is about 27%), and at 40k steps it reaches 9.87 FID vs 22.26 for unfolded Muon and 51.31 for Adam.
Does Muon-C improve image classification?
Yes, modestly at convergence. On ImageNet-100 after 100 epochs it beats unfolded Muon by +0.58 pp (ResNet-34), +0.60 pp (ResNet-50), and +0.26 pp (ConvNeXtV2-T) top-1, with the largest gains appearing early in training. The paper reports 3-seed means but no significance tests.
Is code available for Muon-C?
The paper's text does not include a code or repository link. It does fully specify the algorithm (Algorithm 1), Newton-Schulz coefficients, parameter routing, hyperparameters, and seeds, so reimplementation is possible but requires nontrivial GPU compute (e.g., 400k-step U-Net training).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure panels (Figs. 5–10) appear in the extraction only as axis-label text; trajectory values cited in this review are those quoted in the prose and tables, and exact curve points between checkpoints cannot be independently verified.
- codeAvailable=false reflects the verified absence of any code/repository statement in the full provided text (abstract through references), not truncation.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.