Rank and computation of the pathlifting Jacobian of a DAG ReLU network
stat.ML · cs.LG
Quick verdict
Sound elementary re-proof of rank(∂θΦ)=d−h for DAG ReLU nets plus a backprop-free einsum computation; modest novelty, weak timing baseline.
Summary
This paper studies the Jacobian of the pathlifting map Φ of a DAG ReLU network, which maps the d parameters to the P products of weights along input-to-output paths. It contributes: (1) a self-contained, elementary proof that rank(∂θΦ) = d − h, where h is the number of hidden nodes, recovering a result of Marcotte et al. (ICLR 2026) whose original proof required a graph-theoretic theorem; and (2) the factorization ∂θΦ = diag(Φ)Bdiag(1/θ), where B is the 'skeleton matrix', a P×d binary matrix whose (i,j) entry indicates whether path i uses parameter j. The proof runs an induction on the hidden-node count: after reordering rows and columns around the topologically-last hidden node u, column operations empty the column of the edge from u to a downstream output, and row operations reduce the remainder to the skeleton matrix of a network with h−1 hidden nodes; two worked examples (Sections 3.1–3.2, eqs. 11–22) illustrate every step. Since B is parameter-independent it is computed once, and the Jacobian becomes a single einsum. Experiments on small feed-forward nets (L≤3 layers, w≤8 units, CPU float32) report large speedups over a per-path autograd loop. Main limitations: the headline theorem and Proposition 2 were already known; the timing baseline is a naive autograd loop with no vectorized jacrev/vmap comparator; and scalability is bounded by the exponential path count and memory for Φ and B, acknowledged only in Remark 2.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- For a DAG ReLU network with d parameters and h hidden nodes, rank(∂θΦ) = d − h almost surely (Theorem 1 + Corollary 3), proved by elementary induction on the skeleton matrix.
- The pathlifting Jacobian factorizes as ∂θΦ = diag(Φ)·B·diag(1/θ) (Proposition 2), enabling computation as a single matrix product without backpropagation.
- The skeleton matrix B equals the pathlifting Jacobian at θ = 1d (Corollary 1) and has rank #E − #H (Theorem 1).
- On small feed-forward networks the skeleton-matrix computation is dramatically faster than a per-path autograd loop (fig. 5), with B amortized since it is parameter-independent.
Strengths
Elementary, self-contained proof of a known-but-technical result
Theorem 1 (rk(B) = #E − #H) is proved by induction on the hidden-node count using only row/column operations and a topological sort (Sections 3 and A), replacing the circuit-basis graph theorem of Gleiss et al. (2003) used by Marcotte et al. [2026]; the two fully worked examples (eqs. 11–22) print every intermediate matrix, making the argument hand-checkable.
Actionable computational corollary
Proposition 2, ∂θΦ = diag(Φ)Bdiag(1/θ), turns the Jacobian into a single einsum (Algorithms 3/7); B is θ-independent so it is precomputed once (Section 4), and Corollary 1 identifies B = ∂θΦ(1d), cleanly unifying the theory and the implementation.
Clean new proofs via first-order expansion
Complete implementation detail
Honest scoping of applicability
Weaknesses
Limited headline novelty
Weak experimental baseline
Tiny evaluation scale, no memory data
Editorial and rigor blemishes
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Preliminary Review: Rank and computation of the pathlifting Jacobian of a DAG ReLU network
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 4 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the rank of the pathlifting Jacobian of a ReLU network?
For a DAG ReLU network with d parameters and h hidden neurons, the Jacobian of the pathlifting map has rank d − h, almost surely with respect to any absolutely continuous parameter measure. This was first shown by Marcotte et al. (ICLR 2026) using a graph-theoretic theorem; this paper re-proves it by elementary induction on the skeleton matrix.
How can the pathlifting Jacobian be computed without backpropagation?
Using the factorization ∂θΦ = diag(Φ)·B·diag(1/θ), where Φ is the vector of path products and B is a fixed binary P×d matrix (the 'skeleton') marking which parameters appear in which path. Since B does not depend on the parameters it is computed once at initialization, and the Jacobian is then a single matrix product — for example one torch.einsum call.
What is the skeleton matrix of a neural network?
It is a P×d binary matrix with one row per input-to-output path and one column per parameter, where entry (i,j) equals 1 if and only if path i contains parameter j. The paper shows it equals the pathlifting Jacobian evaluated at the all-ones parameter vector, and that its rank equals the number of edges minus the number of hidden nodes.
Is the d − h rank result new?
No. The rank result rk(∂θΦ) = d − h was proven earlier by Marcotte, Peyré and Gribonval (ICLR 2026) via upper and lower bounds involving a graph-theoretic theorem. The novelty here is an elementary, self-contained induction proof, the explicit link to the skeleton matrix, and a practical, code-backed recipe for computing the Jacobian without backpropagation.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures are available only through captions and inline matrix notation in the extracted text; the actual plots (figs. 1, 5, 7) could not be visually inspected.
- Hyperlinks (the 'GitLab' code repository) do not survive text extraction, so code availability is taken from the paper's statements rather than verified.
- The extracted text appears complete (through Appendix B and references), but equation-heavy passages lost layout, so the block-matrix argument in Appendix A was assessed from the linearized notation.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.