arXiv paper2026-09-18· machine learning / deep learning optimization· method paper with empirical evaluation

COREM: Cosine-Relation Momentum Reshaping with Stateful Writeback

cs.LG

Quick verdict

COREM reshapes momentum via cosine relations; competitive with Muon at toy scale (better CIFAR-10, par on enwik8) but scope and ablations limit claims.

Summary

COREM is an optimizer-state transformation for matrix-valued momentum. It splits the momentum into row-wise update units, builds a signed cosine-relation operator C=OffDiag(V̂V̂ᵀ) scaled by its max-row absolute sum, reshapes momentum via the residual transform V̂−ηCV̂, restores the Frobenius norm, and writes the result back into the momentum buffer, coupling spectral reshaping with momentum memory. Compared with Muon on CIFAR-10 (MLP) and enwik8 (Transformer), COREM reaches better final validation accuracy on CIFAR-10 (61.36% vs 59.74%) with smaller seed spread, while enwik8 is statistically indistinguishable. Spectral diagnostics indicate increased entropy effective rank and reduced top-10 singular-energy concentration. Ablations support stateful writeback and relation-operator normalization, and the transform costs roughly 13.3% of Muon's five-iteration Newton–Schulz FLOPs. Limitations: only two small benchmarks, three seeds without significance testing, both ablations conflate hyperparameter changes or use mismatched grids, no AdamW-style baseline or wall-clock comparison, and code availability is unstated.

Review complete

Assessment at a glance

machine learning / deep learning optimization

risk flags identified

● high 1

● medium 3

● low 2

Reproducibility readinessmedium

Code & data availablenot stated

Review confidencemedium

AI text signal0/20 passages flagged

Deep research viableCode-reproducible

4 strengths · 5 weaknesses · 5 author questions

AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.

Levels reflect verification complexity — not a quality score. Evolve never rates your paper.

Key claims under review

Relations among row-wise units of matrix-valued momentum can be exploited to reshape the optimizer state usefully (§1, §3).
COREM beats Muon in final CIFAR-10 validation accuracy (61.36% vs 59.74%, Table 1) and matches it on enwik8 (Table 2), with faster mid-to-late-stage improvement but slower early progress (§4.3).
Stateful writeback of the transformed momentum is a key component: tuned COREM reaches 61.36% vs 58.61% without writeback on CIFAR-10 (Table 3).
Relation-operator normalization is required for stability: the unnormalized variant diverges in 21 of 28 swept configurations on enwik8 (§4.4.2).
The COREM transform costs about 13.3% of Muon's five-iteration Newton–Schulz transformation FLOPs for square states (§3.5).

Strengths

Clean, reusable abstraction with a concrete instantiation
§3.2 formalizes a unit–relation–transform pipeline; COREM is one instantiation (rows as units, signed cosine relations), and the abstraction explicitly invites alternative relation functions — a genuinely compositional framing rather than a one-off trick.
Closed-form spectral control argument
§3.3 derives ||C̃t||∞-normalization so eigenvalues lie in (−1,1), bounding per-mode gains to [1−η, 1+η] (§3.4); the unnormalized ablation (§4.4.2, 21/28 configurations diverge) empirically corroborates the normalization's role.
Honest, precisely-scoped empirical claims
Diagnostics beyond loss curves

Weaknesses

Toy-scale evaluation with a single baseline
Both ablations are confounded
Weak inferential statistics
FLOPs comparison uses a convenient Muon configuration
Spectral comparison is partly self-referential

Full Review Report

$1.99 · one-time

Unlock the complete preliminary analysis Evolve produced for this paper.

Report preview

COREM: Structured Preliminary Review

Scope & Claims

COREM proposes a unit–relation–transform abstraction (§3.2) for matrix-valued optimizer states and instantiates it as a momentum-reshaping rule: orient the momentum so rows are update units, build signed cosine relations C=OffDiag(V̂V̂ᵀ), normalize by max-row absolute sum ρt, reshape via V̂−ηCV̂, restore the Frobenius norm, and — distinctively — write the transformed state back into the momentum buffer, so the recurrence becomes Mt=T(μMt−1+Gt) (§3.1). The central claims: (1) relations within optimizer states are exploitable structure; (2) COREM beats Muon on CIFAR-10 final v

The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.

All 4 strengths & 5 weaknesses, analyzed in detail
6 risk flags with severity, section & reviewer questions
Methods & statistical evidence assessment
5 questions a peer reviewer would ask
Full six-dimension evaluation report & recommendations

One-time purchase · this paper only

One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.

Frequently asked questions

What is COREM and how does it differ from Muon?

COREM is an optimizer method that reshapes matrix-valued momentum using cosine relations between row-wise 'update units'. It builds a signed cosine-similarity operator, rescales momentum through a residual transform, and writes the reshaped state back into the momentum buffer. Muon instead orthogonalizes the momentum with repeated Newton–Schulz iterations. COREM needs only about 13.3% of the transformation FLOPs of five-iteration Muon for square matrices, trading some early-training speed for stronger late-stage improvement.

Does COREM beat Muon?

In the paper's experiments, COREM reaches better final validation accuracy on CIFAR-10 with an MLP (61.36% vs 59.74% over three seeds) but only comparable results on enwik8 character-level language modeling, where Muon is slightly better on validation loss/BPB and COREM is slightly better on accuracy. Muon progresses faster early in training while COREM improves more in mid-to-late stages. No significance tests are reported, and both evaluations are small scale.

What are the main limitations of the COREM paper?

Evaluation covers only two small benchmarks (a 256×256 MLP on CIFAR-10 and a 4-layer, 192-hidden Transformer on enwik8) against a single baseline; each configuration uses three seeds without significance testing; the writeback ablation changes the learning rate and relation strength at the same time; no AdamW baseline or wall-clock timing is provided; and no code release is mentioned.

Is COREM code or data publicly available?

The paper text does not mention any code or data release. Reproduction would require reimplementing the method from the equations in Section 3 (the transformation itself is short) on public datasets such as CIFAR-10 and enwik8.

About this review

This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.

The extracted text contains no experimental-setup details beyond §4.1 (epochs, weight decay, schedules, tuning protocol, batch size), so reproducibility judgments rely on incomplete information
Figures 1–5 are only available as text fragments; trajectory shapes and box-plot distributions could not be verified numerically
The FLOPs efficiency claim depends on Muon iteration count (five here); practical implementations vary and no wall-clock numbers exist in the text
No code availability statement appears anywhere in the extracted text

Go deeper on this paper

Wrote a paper? Stress-test it before reviewers do.

Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.

Scan your paper