Learning-Induced Dynamical Transition in Recurrent Neural Networks
cs.LG · cond-mat.dis-nn · nlin.CD
Quick verdict
Non-equilibrium DMFT captures a learning-driven chaos-to-stability transition in RNNs; theory matches simulations but is validated at a single parameter point.
Summary
The paper develops a non-equilibrium dynamical mean-field theory (DMFT) for how slow, feedback-driven learning transforms an initially chaotic recurrent neural network into a stable, task-performing one. The model is a random RNN (tanh units, SCS chaotic regime) whose low-dimensional readout is fed back through random projections; readout weights adapt slowly via an error-driven rule with learning rate α. Using an MSRJD path integral, the author derives a closed self-consistent system for the two-time correlation function and the mean output, then exploits the α ≪ 1/τ_c timescale separation: near-diagonal dynamics obey a deformed SCS equation interpretable as a particle in a slowly deforming effective potential, while off-diagonal correlations follow a slow plateau. At a critical normalized feedback ŷ_c satisfying the SCS marginal-stability condition (ŷ_c = 0.2 for g = 1.3), the potential well collapses — a bifurcation marking the chaotic-to-stable transition — and the critical time t_cr depends on g and α. Simulations (N = 5000, 33 realizations, α = 0.007, A = √N) show good agreement with DMFT predictions for output trajectories, diagonal correlations (rising from the SCS value 0.23), and off-diagonal slices after aligning runs by their realization-dependent learning onset times. Main limitations: validation at a single parameter point, stacked approximations with unquantified error, post-hoc onset-time alignment, and no quantitative error metrics or code release.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Slow, error-driven learning of the readout generates an evolving effective feedback strength that acts as an internal dynamical control parameter, driving the network from chaotic to stable dynamics without any externally tuned parameter.
- The chaotic-to-stable transition is a bifurcation of the non-equilibrium DMFT solution occurring at a critical normalized feedback ŷ_c that satisfies the SCS marginal-stability condition (ŷ_c = 0.2 for g = 1.3) and is independent of the learning rate α.
- The time at which the transition occurs, t_cr, depends on g and α and is quantitatively predicted by solving the derived learning trajectory (Eqs. 5.6-5.8).
- DMFT predictions for output trajectories, diagonal correlations, and off-diagonal two-time correlations agree quantitatively with N = 5000, 33-realization simulations after aligning realization-dependent learning onset times.
Strengths
First-principles non-equilibrium DMFT for the full learning trajectory
Unlike prior analyses of only the final trained state (e.g., Clark et al. [7] varying an external feedback parameter), Section 3 (with Appendix A) derives via MSRJD disorder averaging a closed self-consistent system — Eqs. 3.9, 3.12, 3.13 — for the mean output and the full two-time correlator, following the genuinely non-stationary training process.
Analytic identification of the transition mechanism tied to classical results
Section 4 recasts near-diagonal dynamics as a particle in a slowly deforming effective potential (Eq. 4.3); the well-collapse condition (Eqs. 4.4-4.6) recovers the SCS marginal-stability condition 1 = g²∫Dz φ′²(√u_c z), linking the learning-driven bifurcation to the 1988 chaos boundary. The ŷ_c = 0.2 value at g = 1.3 being α-independent while t_cr depends on (g, α) is a sharp, falsifiable distinction.
Complete multi-regime trajectory prediction, not just endpoint statistics
Validation across complementary observables with appropriate diagnostics
Weaknesses
All simulations sit at a single parameter point
Stacked approximations with unquantified combined error
Post-hoc onset-time alignment may inflate apparent agreement
No code or data availability, and visual-only validation
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is a single-author theory paper at the interface of statistical physics and machine learning, developing a non-equilibrium dynamical mean-field theory (DMFT) for a recurrent neural network (RNN) trained by slow, error-driven adaptation of its readout weights in a FORCE-style closed-loop architecture. Three central claims: (i) learning generates an internal, evolving control parameter — the normalized mean output ŷ(t) — that drives the network across a chaotic-to-stable bifurcation without any externally tuned knob; (ii) the critical feedback ŷ_c satisfies the Sompolinsky-Crisanti-Sommers (SCS) marginal-stability condition and is learning-rate independent (ŷ_c = 0.2 at g = 1.3), whereas the critical time t_cr depends on g and α; (iii) the full learning trajectory of output and two-time correlations is quantitatively predicted and matches N = 5000 simulations. The paper positions itself against Clark et al. [7], who varied an external feedback parameter and studied asymptotic states only.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does this paper show about how neural networks learn?
It provides a physics theory (non-equilibrium dynamical mean-field theory) explaining how slow, feedback-based learning pushes an initially chaotic recurrent neural network through a sharp transition into a stable dynamical state. Learning gradually builds structured feedback that deforms an effective landscape; once the feedback reaches a critical strength, chaotic fluctuations collapse and the network produces stable, task-consistent output. The theory quantitatively predicts the network's output trajectory during training, matching simulations of a 5000-neuron network.
What is the critical feedback strength at which the transition happens?
For recurrent gain g = 1.3, the transition occurs when the normalized network output — which equals the effective feedback strength, with feedback variance set to 1 — reaches ŷ_c = 0.2. This value satisfies the same marginal-stability condition as the classic Sompolinsky-Crisanti-Sommers chaos boundary. Notably, it does not depend on the learning rate; only the time at which it is reached during training depends on the learning rate α.
Was the theory validated by simulations, and how thoroughly?
Yes, but at a single configuration: a network of N = 5000 neurons with g = 1.3, constant target A = √N, and learning rate α = 0.007, across 33 random-wiring realizations. After aligning runs by their realization-dependent learning onset times, the mean output trajectory, diagonal correlations (rising from the chaotic value 0.23), and off-diagonal correlation slices all agree well with the theory. No other parameter values are tested, no error metrics are reported, and no code is provided.
How is this different from earlier work on chaos suppression in trained RNNs?
Earlier studies (e.g., Clark et al., cited as bioRxiv 2026) introduced an external control parameter and characterized the final trained state. This paper shows that the learning process itself generates the evolving control parameter — the growing learned feedback — and follows the entire non-stationary training trajectory using a two-time correlation formalism, separating a learning-rate-independent critical feedback from a learning-rate-dependent critical time.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-7 are available only as captions in the extracted text; the visual quality of the claimed DMFT-simulation agreement cannot be independently verified.
- No data or code availability statement exists in the manuscript, so reproducibility is assessed from the described methods alone.
- Reference [7] (Clark et al., bioRxiv 2026) is central to the novelty positioning but could not be checked against its content.
- The text appears complete (through references), so truncation is not a concern, but extracted equations contain minor formatting artifacts.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.