MpSub: A Momentum $p$-Dimensional Subspace Trust-Region Method for Derivative-Free Fine-Tuning of Large Language Models
Yuyang Wang, Haoyu Yao, Pengcheng Xie
cs.LG · math.OC
Quick verdict
ZO trust-region method drops MeZO's learning rate with a.s. convergence proof; matches tuned MeZO on two small OPTs, but evidence spans one 56-example dataset.
Summary
The paper proposes MpSub, a derivative-free trust-region method for full-parameter LLM fine-tuning that removes the need for a hand-tuned learning rate. Each iteration searches a p-dimensional subspace spanned by the normalized most-recent accepted step (momentum) and p-1 fresh Gaussian directions; a central-difference subspace gradient defines a linear trust-region model, and the radius - serving as both finite-difference perturbation and step bound - adapts by predicted-versus-observed reduction. For LLMs, all evaluations share one minibatch and directions are regenerated from seeds in place, keeping memory at inference level. For smooth deterministic objectives under unorthogonalized Gaussian frames, the authors bound finite-difference error, quantify captured gradient energy, and prove the gradient norm converges to zero almost surely under a safeguarded radius rule. Under a matched budget of 8,400 training forward passes on CommitmentBank, MpSub reaches mean test accuracy 0.673 (OPT-125M) and 0.690 (OPT-350M) versus 0.685 for dev-tuned MeZO at both sizes, with identical preset hyperparameters across models and a stable plateau across a 300-fold range of initial radii. Limitations: evidence comes from one 56-example test set, three seeds, two small models, and a MeZO-only comparison; the convergence theory assumes a deterministic objective and a safeguarded update not used in the experiments; the subspace-dimension ablation is single-seed.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- MpSub eliminates the learning rate by letting one adaptive trust-region radius serve as both the finite-difference perturbation and the step bound (Sections 2.2-2.3).
- For smooth deterministic objectives with Lipschitz gradient, under unorthogonalized Gaussian directions and a safeguarded radius update, the gradient norm converges to zero almost surely (Theorem 3.8).
- Under a matched 8,400-forward-pass budget on CommitmentBank, MpSub with identical preset hyperparameters attains accuracy comparable to dev-tuned MeZO: test 0.673 vs 0.685 on OPT-125M and 0.690 vs 0.685 on OPT-350M (Table 4.1).
- MpSub is robust across a 300-fold range of initial radii (test accuracy 0.655-0.678 on OPT-125M) while MeZO degrades sharply off its best learning rate (Section 4.3).
Strengths
Genuine learning-rate elimination, demonstrated empirically
Section 4.3 shows MpSub holds test accuracy in 0.655-0.678 across a 300-fold range of initial radii (1e-3 to 3e-1) on OPT-125M, while MeZO loses 0.048 at one order below and 0.084 at one order above its best rate (1e-6). The same preset hyperparameters (p=20, delta_0=1e-1) work at both model sizes (Table 4.1).
Convergence theory for the actual unorthogonalized Gaussian frame
Theorem 3.8 proves the gradient norm goes to zero almost surely without assuming orthogonal directions, building on the finite-difference bound (Lemma 3.2), Gaussian gradient-capture in distribution (Lemma 3.3, chi-squared argument), radius summability (Lemma 3.5), and a careful three-point argument over excursions (Theorem 3.6 plus Lemma 3.7).
Memory-honest LLM implementation
Honest, symmetric reporting of results
Well-motivated decoupling of acceptance from radius update
Weaknesses
Extremely narrow empirical evidence
MeZO is the only baseline
Theory-practice mismatch on the safeguarded update
Statistical reporting too weak for the effect sizes involved
Per-step cost asymmetry is not discussed against wall-clock
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
MpSub is a method paper at the intersection of derivative-free optimization (DFO) and LLM fine-tuning. The central claim is architectural: replace MeZO's hand-tuned learning rate with a trust-region radius that simultaneously controls the central-difference perturbation and the step bound (Sections 2.2-2.3). This is supported by (i) an almost-sure first-order convergence theory for smooth deterministic objectives under unorthogonalized Gaussian frames (Theorem 3.8) and (ii) matched-budget experiments against MeZO on CommitmentBank with OPT-125M/350M (Section 4, Table 4.1). A secondary claim is robustness: stable accuracy across a 300-fold range of initial radii (Section 4.3). To the authors' credit, the claims are scoped honestly - the abstract and Section 5 say comparable, not superior, and explicitly disclaim convergence for the stochastic minibatch setting.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What problem does MpSub solve compared to MeZO?
MeZO is a zeroth-order (forward-pass-only) fine-tuning method that needs a learning rate retuned for each model and task, and a single random direction carries very little gradient information in billion-parameter spaces. MpSub replaces the learning rate with an adaptive trust-region radius that also sets the finite-difference perturbation, and searches a p-dimensional subspace combining the previous accepted step (momentum) with fresh random directions. In the paper, one preset hyperparameter set worked unchanged on both OPT-125M and OPT-350M.
Does MpSub beat MeZO?
Not consistently - the honest reading is parity with less tuning. Under a matched 8,400-forward-pass budget on CommitmentBank, MpSub scored 0.673 vs MeZO's 0.685 test accuracy on OPT-125M (MeZO slightly better) and 0.690 vs 0.685 on OPT-350M (MpSub slightly better), with three seeds and differences inside seed noise. The clear demonstrated advantage is robustness: MpSub stays within 0.655-0.678 test accuracy across a 300-fold range of initial radii, while MeZO drops 0.048-0.084 when its learning rate moves one order of magnitude from the best value.
What convergence guarantee does MpSub have?
For smooth deterministic objectives with Lipschitz-continuous gradient, using Gaussian directions without orthogonalization and a safeguarded radius update (expansion additionally requires the estimated gradient norm to exceed kappa times the radius), the gradient norm is proven to converge to zero almost surely. The LLM experiments resample minibatches every iteration and run the unsafeguarded update, so the guarantee does not directly apply to the experimental configuration - a limitation the authors acknowledge.
Is code or data available for MpSub?
The preprint text does not provide a code release link. The authors state their experimental infrastructure builds on the public MeZO and LOZO codebases, and the paper gives complete hyperparameters, the direction-seed formula, and standard CommitmentBank splits, so a reimplementation from the paper is feasible; the data (SuperGLUE CommitmentBank, OPT models) is public.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 4.1 and 4.2 are present only as axis labels and captions in the extraction; underlying curve values could not be read, so all quantitative statements here rely on Tables 4.1/4.2 and in-text numbers.
- The abstract in the task metadata is cut mid-sentence, but the full abstract is present in the embedded paper text; no truncation of the paper body was detected (the text ends with the complete reference list).
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.