On the Abundance of Critical Points of the t-SNE Energy
cs.LG · math.AP · math.ST
Quick verdict
Constructs infinite families of t-SNE critical points via symmetry pairs for symmetric data, but the headline theorem hinges on two unproven conjectures.
Summary
This paper takes rigorous first steps toward explaining why t-SNE exhibits topology breaking and spurious clustering. The authors formulate a generalized t-SNE energy over couplings on Riemannian manifolds (Section 2.3), unifying the original discrete energy, the coupling limit of Auffinger-Fletcher (2023), and the map-based continuum limit of Murray-Pickarski (2024). After reproducing classic pathological examples (spaghetti plots, grid twisting, Swiss roll; Section 3.1), they develop symmetry-invariant couplings for pairs of finite isometry groups (Section 4), including symmetry pairs that are 'mutually exclusive up to constants.' They prove the continuum gradient flow is well-posed (Proposition 7), preserves symmetry invariance (Theorem 4), and converges to symmetry-invariant critical points when embedding moments stay bounded (Theorem 11). A second-variation argument shows constant embeddings are not local minima among symmetric maps in Euclidean settings (Proposition 14), yielding the headline result: for radially symmetric data there exist infinitely many distinct critical points (Theorem 16) - conditional on two open conjectures (moment bounds and persistence of map-induced couplings). Numerical experiments (Examples 20-23) show vanilla gradient descent preserves symmetry to about 1e-15 while openTSNE breaks it (about 1e-2 to 1e-1) yet attains lower energy (Table 2). Main limitations: the critical points are not shown to be stable minima, both conjectures remain open, and empirical validation uses small single-run datasets.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- The gradient flow of the generalized t-SNE energy over couplings is well-posed and preserves invariance under paired finite isometry groups (Proposition 7, Theorem 4).
- For every admissible symmetry pair there exists a symmetry-invariant critical point, given compactness of Y or a moment bound (Theorem 11).
- In Euclidean settings, the constant embedding is not a local minimum of the t-SNE energy within the class of symmetry-invariant maps (Proposition 14).
- Conditional on two conjectures, radially symmetric feature data admits infinitely many pairwise-distinct critical points of the t-SNE energy (Theorem 16).
- Exact gradient descent preserves numerical symmetry to ~1e-15 while approximate optimizers (openTSNE with Barnes-Hut) break symmetry yet attain lower energy (Examples 20-21, Table 2).
Strengths
Unifying energy framework
Section 2.3 (eqs. 2.4-2.6) formulates one generalized energy over couplings on Riemannian manifolds that provably recovers the discrete t-SNE energy, the Auffinger-Fletcher (2023) coupling limit, and the Murray-Pickarski (2024) map-based limit, with concrete kernel examples (Euclidean, torus, sphere, hyperbolic; Examples 1-4).
Rigorous well-posedness for a nonstandard flow
Proposition 7 proves existence/uniqueness for the gradient flow (2.12), which moves only the Y-marginal of a fibred coupling and so escapes standard Wasserstein theory; global extension rests on the a priori bound 1/I(πt) ≤ C·e^(C√t) derived in Lemma 5 from the KL lower bound (5.1).
Clean symmetry mechanism with quantitative numerics
Honest accounting of conditionality
Weaknesses
Headline result is conjecture-conditional
Critical points are not shown to be minima or attractors
Single-run, small-scale numerics without uncertainty
Theory-practice gaps acknowledged but unresolved
Cross-reference defects
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper targets a long-standing empirical puzzle: t-SNE embeddings routinely break topology ("spaghetti plots", Example 5) and fabricate clusters (grid twisting, Example 6; Swiss roll, Example 7). Rather than analyzing specific minimizers, the authors attack the landscape: for a generalized t-SNE energy over couplings on Riemannian manifolds (Section 2.3, eqs. 2.4-2.6) that subsumes the discrete t-SNE energy, the coupling limit of Auffinger-Fletcher (2023), and the map-based limit of Murray-Pickarski (2024), they show that symmetric data distributions admit infinitely many distinct critical points. The mechanism: pairs of finite isometry groups (GX, GY) tied by a group isomorphism define symmetry-invariant couplings; gradient flow preserves this invariance; each admissible pair yields a critical point; and "mutually exclusive up to constants" pairs yield distinct ones. Importantly, the abstract reads more unconditional than the theorems: Theorem 16 requires Conjectures 9 and 15, as Section 1 openly states.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does this paper prove about t-SNE?
It proves that when the input data distribution has suitable symmetries (e.g. radial symmetry), the t-SNE energy has many critical points: paired finite isometry groups acting on data and embedding spaces are preserved by gradient descent, each valid pair yields a symmetry-invariant critical point, and (conditionally on two stated conjectures) there are infinitely many pairwise-distinct ones. This offers a rigorous mechanism behind t-SNE's well-known topology-breaking and spurious-cluster artifacts.
Are the main results unconditional?
No. The core infrastructure results - well-posedness of the gradient flow and preservation of symmetry along it - are unconditional. However, the headline theorem (infinitely many distinct critical points for radial data) depends on two open conjectures: that embedding moments stay uniformly bounded over time, and that gradient flows starting from map-induced couplings remain map-induced.
Does the paper show these critical points cause the clusters seen in real t-SNE plots?
Not directly. The paper proves such critical points exist and reproduces pathological examples numerically, but it does not prove the constructed points are stable local minima or track which minimizers gradient descent reaches from random initialization. The authors explicitly leave stability open, and their own energy tables suggest symmetric critical points could be saddles.
Is code available for the numerical examples?
Yes. The authors state the code is on their public GitHub repository (github.com/nak-har/abundance_of_critical_points_tsne), covering the exact-gradient and openTSNE experiments, including the symmetry-defect measurements reported in their tables.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures could not be visually inspected (image content is unavailable in text extraction); assessments of the plots rest on captions and the reported numerical values.
- Mathematical notation is partially garbled by PDF extraction (inline equations, sub/superscripts), so symbol-level details of proofs were read with care but not character-verified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.