Bias-Induced Crossover in Absolute Capacity of Dense Associative Memory
Yuto Sakurai, Takeaki Shimokawa, Kazunori Iwata, Kazushi Mimura
cond-mat.dis-nn · cs.LG
Quick verdict
Bias cuts dense-memory absolute capacity from N^(n-1)/ln N to O(N^(n/2)) near q=1/2; analysis is careful, but Gaussian-tail claims lack rigorous validation.
Summary
This theoretical paper analyzes how bias in centered binary patterns changes the absolute capacity of dense associative memory with polynomial order-n interactions, under the Krotov-Hopfield single-site criterion P_error=1/N. A signal-to-noise analysis with a conditioned-Gaussian approximation (exact binomial finite-size terms retained) yields: at q=1/2 the known K_max ~ N^(n-1)/(2(2n-3)!! ln N); for fixed q<1/2, K_max drops to O(N^(n/2)) for even n>=4 and O(N^((n+1)/2)) for odd n>=5, while n=3 keeps O(N^2/ln N). The mechanism is a bias-dependent crosstalk mean that destabilizes sites carrying the frequent value -q; asymptotic matching gives crossover width 1-2q=O(ln N/N^(floor(n/2)-1)). Simulations with explicitly generated patterns (n=3,4 up to N=400; n=5 only N=100) agree with finite-size theory in the moderate-bias regime but deviate at large X_N and small q. An activity-dependent control potential cancelling the conditional crosstalk mean restores the N^(n-1)/ln N scaling, but only within the conditioned-Gaussian approximation and without simulations of the controlled model. Key limitations: the 1/N tail estimate lacks a Cramér-type moderate-deviation proof, and the sparse limit q->0 is left open.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
4 strengths · 4 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- For fixed bias q<1/2, the absolute capacity of order-n dense associative memory drops from N^(n-1)/ln N to O(N^(n/2)) (even n>=4) or O(N^((n+1)/2)) (odd n>=5), while n=3 retains O(N^2/ln N).
- The reduction originates from a bias-dependent conditional crosstalk mean that destabilizes sites carrying the more frequent value -q, so P_error is asymptotically (1-q)p_- (Eq. 37).
- The unbiased and fixed-bias regimes are connected by a crossover of width 1-2q = O(ln N/N^(floor(n/2)-1)), estimated by asymptotic matching (Eq. 42).
- An activity-dependent control potential cancelling the conditional crosstalk mean restores capacity N^(n-1)/(8(2n-3)!! q(1-q) ln N) for any fixed 0<q<1/2 within the conditioned-Gaussian approximation (Eq. 64).
Strengths
Mechanism is traced to a specific, checkable term
Appendix A derives the leading conditional crosstalk means (Eq. A20/A21 vs A23) and shows the frequent-value (-q) mean is reduced by a bias-dependent term of order KN^(n-2)/2 (even n) or KN^(n-3)/2 (odd n); Appendix B then proves p+/p- -> 0 so Eq. (37) holds for all n>=3, making the crossover attribution internally coherent.
Finite-size theory retains exact discrete structure rather than crude Gaussianization
Section III A keeps the exact binomial distributions of M, X_nu, Y_nu and of a single crosstalk contribution (Eqs. 22-23), applying the Gaussian approximation only to the sum; the variance uses the full law of total variance (Eq. A34) including the K^2 common-activity covariance term, which Eq. (A38) shows is subleading but non-negligible at finite N.
Consistency checks across regimes
Honest scoping of criterion and limitations
Weaknesses
Central tail claims rest on a Gaussian approximation at probability 1/N
The control-potential remedy is never simulated
Simulation coverage is thin on exactly the regimes that matter
No code or data availability statement
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the difference between absolute capacity and storage capacity of a Hopfield network?
Storage capacity (e.g., Amit-Gutfreund-Sompolinsky's 0.138N) counts how many patterns can be retrieved with a finite fraction of bit errors, using a macroscopic retrieval overlap. Absolute capacity is stricter: it asks how many patterns can be stored such that a random single neuron flip almost surely raises the energy, i.e., the stored patterns are exact energy minima. For the standard unbiased Hopfield model this stricter criterion gives about N/(2 ln N) patterns (McEliece et al.; Krotov-Hopfield).
Why does bias in the stored patterns reduce the capacity of dense associative memory?
Centering makes the pattern mean zero, but the two component values (+ (1-q), rare; -q, frequent for q<1/2) are not symmetric. The crosstalk noise contribution to a site's energy gap has a different mean depending on which value the site carries, and this bias-dependent mean grows with the number of stored patterns and destabilizes the frequent-value sites first. The paper shows this cuts capacity from N^(n-1)/ln N to roughly N^(n/2) for interaction order n>=4, with the two regimes connected by a crossover of width 1-2q ~ ln N/N for n=4,5.
Can the capacity loss from biased patterns be fixed?
Within the paper's Gaussian approximation, yes: adding an activity-dependent potential to the energy — whose increment equals minus the conditional crosstalk mean at each realized activity level, plus a constant balancing the two error rates — cancels the harmful mean exactly and restores the unbiased capacity order N^(n-1)/ln N for any fixed bias. Importantly, this is a static energy modification requiring no knowledge of which pattern is being tested, but the paper does not simulate this controlled model, so the fix is theoretically supported only.
Is dense associative memory related to modern Hopfield networks and transformers?
Yes. Krotov and Hopfield's dense associative memory with polynomial or exponential interactions is equivalent to the modern Hopfield network framework of Ramsauer et al., whose continuous-state version is mathematically equivalent to the attention mechanism in transformers. This paper stays in the binary-state polynomial setting and analyzes a stricter (error-free, single-site) capacity criterion under biased patterns, complementing thermodynamic retrieval analyses such as the 2026 replica-symmetric study of Albanese et al.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures (1-5) are available only as captions in the extracted text; graphical agreement between theory and simulation could not be independently assessed.
- The embedded text appears complete (all eight sections plus Appendices A-D and references), so main results were verifiable, but equation rendering of double-factorials and sub/superscripts may contain extraction artifacts.
- No code, data, or supplementary-material links appear in the text; reproducibility assessment relies solely on the written simulation protocol.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.