Algebraic Expressivity Certificates for Shallow Polynomial Neural Networks
Sepehr Akbari, Shahrzad Jamshidi
math.AC · cs.LG · math.AG
Quick verdict
Correct elimination-based nonrepresentability certificates for shallow polynomial nets, but theory is largely classical and 67/100 benchmarks time out.
Summary
This paper studies exactly which degree-d homogeneous forms can be written by a bias-free, width-r network with monomial activation z↦z^d. Over C such networks compute sums of r d-th powers of linear forms, whose Zariski closure is the r-th secant variety of the Veronese variety; eliminating the weight variables from the graph ideal therefore yields polynomial equations whose violation is an exact, optimization-independent certificate of nonrepresentability — while their simultaneous vanishing decides only border rank, not exact rank, for d≥3. For quadratics the authors show image equals closure (symmetric matrices of rank ≤ min(r,n)), derive the dimension formula ns − s(s−1)/2, and explain the parameter redundancy via the orthogonal group action. A Macaulay2 pipeline sweeps 100 architectures (n,r ≤ 6, d ≤ 5) under a 3600 s cutoff: 33 complete, 21 returning zero ideals and 12 nonzero, all consistent with the Alexander–Hirschowitz theorem; it recovers the binary-quartic catalecticant cubic and shows the (3,3,2) elimination output equals the 20 classical catalecticant minors over Q. A sphere experiment derives an exact population loss floor of 1/6 for rank-2 PSD quadratic approximation of x1²+x2²+x3², attained at eigenvalues (5/4, 5/4, 0); a single-seed SGD run reaches ≈0.16. Limitations: 67/100 timeouts, largely classical theory, single-seed numerics, and no public repository URL stated.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 1
● low 3
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Eliminating weight variables from the graph ideal of a width-r z^d network yields defining equations of the Veronese secant variety, so a violated equation is an exact, optimization-independent certificate of nonrepresentability at that width.
- For quadratic networks, image equals closure: the representable class is exactly the complex symmetric matrices of rank at most min(r,n), with dimension ns − s(s−1)/2 and an O(r,C)-orbit explanation of the rn − r(r−1)/2 parameter redundancy.
- The Macaulay2 pipeline's 33 completed cases (out of 100 within the 3600 s cutoff) all match the Alexander–Hirschowitz/zero-nonzero predictions, and it recovers the binary-quartic catalecticant cubic plus the 20 ternary-cubic secant minors, with the (3,3,2) equality verified exactly over Q.
- For a rank-2 positive-semidefinite quadratic approximating x1²+x2²+x3² under the uniform measure on the 2-sphere, the population loss floor is exactly 1/6, attained at eigenvalues (5/4, 5/4, 0).
Strengths
Honest image/closure distinction anchors the certificate logic
Corollary 4 and Remark 11 precisely delimit what elimination proves: a violated equation certifies nonmembership in the border-rank locus, while simultaneous vanishing decides only border rank for d≥3, illustrated concretely by x₁³x₂ (border rank ≤ 2, Waring rank 4)
Exact arithmetic with verified recoveries of known ideals
The quadratic determinant (eq. 9) expands correctly from eq. (8); the binary-quartic cubic (eq. 12) matches the classical catalecticant expansion; the (3,3,2) elimination output is proven equal to the 20 classical minors via a rank-20 subspace comparison over Q (Prop. 12)
Transparency about computational limits
Detailed reproducibility appendix
Weaknesses
Two-thirds of the benchmark times out
Shallow verification beyond the worked examples
Limited theoretical novelty
Thin empirical component
Code access and citation hygiene
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper addresses exact representability by bias-free shallow polynomial networks f(x) = Σᵢ aᵢ(wᵢᵀx)^d over C, identified (after absorbing a d-th root of each aᵢ) with sums of r d-th powers of linear forms. The image F_{n,d,r} is the symmetric rank-≤r locus; its Zariski closure V_{n,d,r} is the affine cone over the r-th secant variety of the Veronese embedding (Prop. 2). Central claims: (i) eliminating weight variables from the graph ideal yields I(V_{n,d,r}) (Prop. 3), so any violated equation certifies nonrepresentability exactly (Cor. 4) — while simultaneous vanishing decides only border rank for d ≥ 3, illustrated by x₁³x₂ (border rank ≤ 2, Waring rank 4, Remark 11); (ii) for d = 2 image equals closure: symmetric matrices of rank ≤ min(r,n), dimension ns − s(s−1)/2 (Thm 6, Cor. 7), with the rn − r(r−1)/2 parameter loss explained by O(r,C)-orbits (Rem. 8); (iii) a Macaulay2 pipeline realizes this for arbitrary (n,d,r) plus a 100-tuple benchmark; (iv) an exact population loss floor of 1/6 for a rank-2 PSD quadratic approximating x₁²+x₂²+x₃² on S² (Prop. 13).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does this paper actually prove about neural networks?
For bias-free width-r networks with activation z^d over C, the output is a sum of r d-th powers of linear forms. Eliminating the weight variables yields polynomial equations that vanish exactly on the closure of this set (symmetric border rank at most r). If a target function violates any of these equations, it provably cannot be represented at that width — an exact, optimization-independent architectural obstruction.
What is the difference between exact representability and border rank in this paper?
Exact representability means the target equals a sum of r d-th powers of linear forms (Waring rank at most r). Border rank means it is a limit of such sums. These coincide for quadratics but differ in degree 3 and above: for example x1^3*x2 has border rank at most 2 but exact rank 4, so satisfying all secant-variety equations does not guarantee an exact width-r decomposition.
What is the 1/6 loss floor result?
For inputs uniform on the 2-sphere, the paper proves the best rank-2 positive-semidefinite quadratic approximation to x1^2+x2^2+x3^2 has population mean-squared error exactly 1/6, attained when the approximating matrix has eigenvalues (5/4, 5/4, 0). This is an irreducible error caused purely by the algebraic rank obstruction, not by optimization failure.
Is the code and data for this paper available?
Appendix A names concrete Macaulay2 and Python scripts (src/fn_power.m2, script/expressivities.py, script/validate_ternary_cubic.py, script/sphere_pred.py) and says new runs record a repository commit in a JSON metadata header, but the extracted text provides no public download URL. All completed computations are small enough to re-derive independently on CPU.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- PDF extraction garbles some math glyphs (inline binomials) and two Table 3 rows ((3,3,2), (2,4,2)); those column values were cross-read from Table 2, which lists the same entries cleanly.
- No truncation detected: the provided text ends cleanly after Appendix A, and every number cited in this review appears verbatim in the extraction.
- The extracted text names code files and a recorded repository commit but contains no public repository URL anywhere; code access could not be verified from the text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.