Comparing Latent Concept Formation in State Space Models and Transformers via Sparse Autoencoders
Rithin Nagaraj, Rupa Laalasa Oruganti, Prerna Subhashchandra Kunder, Ashwini M Joshi
cs.LG
Quick verdict
SAE study finds 99.98% Mamba–Pythia feature alignment, backing universality, but rests on uncalibrated Jaccard thresholds and a single twin-pair case study.
Summary
This paper asks whether a selective state-space model and a Transformer learn distinct latent features. The authors compare Mamba-130m and Pythia-70m-deduped by decomposing mid-layer residual-stream activations with sparse autoencoders — a pretrained SAELens SAE for Pythia (32,768 features) and a custom L1 SAE for Mamba (12,288 features) — binarizing activations over 10 million Wikipedia tokens, and computing pairwise Jaccard co-activation similarity, assigning each Mamba feature its best-matching Pythia 'twin'. Observed scores span only 0.234–0.412. Using data-derived thresholds, 99.98% of Mamba features (12,285 of 12,288) fall in the 'Aligned' band (J>0.35), two in an intermediate 'Grey Matter' band, and one 'Alien' feature — read as preliminary support for the Universality Hypothesis. Qualitative analysis of the rare divergences finds structural-landmark neurons (document footers, URL separators) and one twin-pair case study in which the Mamba feature is polysemantic (a 'junk drawer' firing on URL slashes, hyphens, and capitalized foreign phrases) while its Pythia twin responds only to table timestamps. Main limitations: thresholds lack null-distribution calibration; dictionaries are asymmetric (16× vs 64× expansion); the polysemanticity claim rests on a single pair; and no code link or Mamba SAE hyperparameters are reported.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 2
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- 99.98% of Mamba-130m mid-layer SAE features (12,285 of 12,288) have a Pythia-70m twin with Jaccard co-activation above 0.35 over 10M Wikipedia tokens, which the authors read as preliminary support for the Universality Hypothesis.
- The few diverging features are monosemantic structural landmarks — document-terminating headers (Feature 6888) and URL separators (Feature 7378) — hypothesized to act as state resets in the recurrent hidden state.
- A single twin-pair case study (Mamba Feature 4454 vs Pythia Feature 4309) suggests Mamba superimposes unrelated syntactic anomalies into polysemantic features while Pythia decomposes them monosemantically.
- Architectural routing (attention vs recurrent state) has negligible impact on core semantic ontology, with divergence confined to structural/syntactic margins.
Strengths
A discrete, token-level complement to prior cross-architecture SAE work
Where Wang et al. [12] used aggregate Max Pairwise Pearson Correlation on the same Mamba-130m/Pythia pairing, this paper's binary co-activation Jaccard over 10M tokens enables localizing the exact token contexts that drive divergence (Sec II.C) — and the examples given are concrete enough to verify.
Unusually candid limitations section
Sec V.D explicitly names the dictionary-asymmetry confound (16× vs 64×) and downgrades the polysemanticity finding to 'hypothesis-generating'; Sec III.E admits the category thresholds are descriptive partitions of the observed data rather than principled boundaries.
Concrete, inspectable divergent features
A useful negative result with reusable machinery
Weaknesses
Circular, uncalibrated thresholds drive the headline number
Max-selection over 32,768 candidates without any null baseline
Asymmetric SAE dictionaries confound both directions of the analysis
Architecture-level claims rest on 3 features and one twin pair
Reproducibility asserted but not demonstrated
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper (arXiv 2609.24440, cs.LG; PES University) asks whether the recurrent state bottleneck of Selective State Space Models produces latent representations distinct from Transformers. It compares Mamba-130m against Pythia-70m-deduped at one mid-depth layer each (Mamba `blocks.12.hook_resid_post` of 24 layers; Pythia `blocks.3.hook_resid_post` of 6 layers), decomposing activations with SAEs over 10 million Wikipedia tokens (GPT-NeoX tokenizer, non-overlapping 1,024-token sequences). Three central claims: (1) 99.98% of Mamba features have a Pythia twin with Jaccard > 0.35 — 'strong universality'; (2) the rare diverging features (2 Grey Matter + 1 Alien = 3 total) are monosemantic structural landmarks acting as state resets; (3) one twin-pair case study (Mamba Feature 4454 vs Pythia Feature 4309) suggests Mamba crams unrelated syntactic patterns into polysemantic 'junk drawer' features. Scope is explicitly one model pair, one layer pair, one corpus.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Do Mamba and Transformer language models learn the same internal features?
According to this paper, largely yes at the layer studied: when mid-layer activations of Mamba-130m and Pythia-70m are decomposed with sparse autoencoders and compared over 10 million Wikipedia tokens, 99.98% of Mamba features had a Pythia counterpart with Jaccard co-activation above 0.35. The authors call this preliminary support for the Universality Hypothesis, while noting the thresholds were derived from the observed data itself and the two SAE dictionaries were of different sizes.
What is the Universality Hypothesis in mechanistic interpretability?
It is the conjecture that neural networks trained on similar data converge on similar internal representations and computational circuits, regardless of architecture or initialization. This paper tests a feature-level version of it by matching sparse-autoencoder features between a state-space model (Mamba) and a Transformer (Pythia) trained on comparable text.
What is a 'junk drawer' neuron in this paper?
The authors' term for a polysemantic Mamba feature (Feature 4454) that fires on several unrelated syntactic patterns — URL slashes, K-12 grade hyphens, and capitalized foreign phrases — while its best-matching Pythia feature (Feature 4309) responds only to table timestamps in Wikipedia markup. They propose this reflects Mamba's fixed recurrent state capacity forcing unrelated patterns to share one feature.
Is it now proven that SSMs and Transformers represent language identically?
No. The study covers one small model pair (70m/130m parameters), one mid layer per model, one corpus (Wikipedia), and asymmetric SAE dictionaries (16× vs 64× expansion). The alignment threshold was chosen from the observed data with no null baseline, and the divergence analysis rests on three features and one case study. The authors themselves present the work as preliminary and hypothesis-generating.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- No distribution figures (e.g., a Jaccard histogram) were present in the extracted text; all distribution-shape statements rest on the three reported summary statistics (min 0.2343, max 0.4116, category counts).
- Possible appendix or supplementary material (e.g., Mamba SAE training details) may have been truncated; the absence of hyperparameters is judged from the text as provided.
- Review is based solely on the authors' reported statistics and quoted activation snippets; no independent verification of the feature examples was possible.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.