Replicating the Geometry of Emotion Representations in a Base Open-Weights Model
cs.CL · cs.AI · cs.LG
Quick verdict
Rigorous replication of Claude's emotion geometry on base Gemma-2-27B: core structure holds, arousal not certified; limited by single coder and one corpus draw.
Summary
This paper replicates the representational core of Sofroniew et al. (2026) — which found that 171 emotion vectors extracted from Claude Sonnet 4.5 form a valence–arousal circumplex — on the base open-weights model Gemma-2-27B. The author regenerates the full 205,200-story Claude-generated corpus to the original design, inherits every disclosed parameter (token-50 pooling, difference-of-means extraction, 50%-variance neutral-PC cleanup, L31 at 2/3 depth), and changes only the subject model. Core results reproduce: PC1 carries 26.7% of variance (original ~27%) and aligns with human pleasure norms at r=0.72 (original 0.81); PC2 carries 13.4% (original ~14%) and aligns with arousal at r=0.67 (original 0.66), but arousal holds only at full emotion-set scale and late depth, so it is not classified as replicated. Extensions locate a sharp L22–26 seam where geometry consolidates and the vocabulary readout switches on, show much of the geometry pre-exists in static token embeddings (arousal excepted, 0.18→0.67), find co-activation on held-out text matching the geometry at r=0.907, and quantify a 52.6% floor of structurally non-conceptual peak tokens. Limitations: the original study released no code or corpora, so comparisons rest on reconstructions; hand-validation used a single coder; the co-activation analysis uses only the 723 top-activating documents; and cross-model differences are confounded with model family, developer, and base-versus-tuned training.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 0 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- The affective circumplex and leading-component variance split of Sofroniew et al. (2026) reproduce on base Gemma-2-27B (PC1 26.7% vs ~27%; PC2 13.4% vs ~14%), with valence aligning to human norms at r=0.72 (original 0.81).
- Arousal aligns at r=0.67 (original 0.66) but holds only at the full 171-emotion scale and late depth, so the authors do not classify it as replicated.
- A sharp L22–26 seam consolidates the geometry, and the vocabulary readout switches on at L23 for structural rather than frequency-driven reasons.
- Much of the emotion geometry already exists in static token embeddings (RSA 0.882 to L31), with arousal the exception (0.179 at embedding vs 0.668 at L31).
- The geometry predicts token-level co-activation on held-out text at r=0.907, and at least 52.6% of rank-1 peak tokens are structurally non-conceptual, a measured floor for max-activation confounds.
Strengths
Unusually explicit replication discipline
Table 1 separates inherited, resolved, and new choices; PCA component signs are fixed by anchor emotions chosen before any comparison; qualitative replication criteria are pre-stated in §1 rather than fitted to results.
Honest negative classification of arousal
Despite r=0.67 vs the original's 0.66, arousal is not certified as replicated because it holds only at full 171-emotion scale and late depth (§5.2, §6.4, Table 2) — a deliberate resistance to overclaiming.
Confound quantification beyond the original
Genuine reconciliation with concurrent work
Measured, not assumed, numerical precision
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is a single-author replication of the representational core (Parts 1–2) of Sofroniew et al. (2026, arXiv:2604.07729), which reported that 171 emotion vectors difference-of-means-extracted from Claude Sonnet 4.5 form a valence–arousal circumplex. The sole targeted change is the subject model: base google/gemma-2-27b (46 layers, analyzed at L31 = 2/3 depth). The author regenerates the full 205,200-story corpus (100 topics × 171 emotions × 12 variants, one Claude regime, 0.09% artifacts stripped) plus 500 neutral dialogues, inherits every disclosed parameter (token-50 pooling, 50%-variance neutral-PC cleanup, k=10, Russell–Mehrabian benchmark), and resolves unspecified steps by disclosed rules (Table 1). Causal/steering and assistant-facing analyses are out of scope. Headline claims: the circumplex recovers (PC1 26.7% vs ~27%; PC2 13.4% vs ~14%); valence aligns with human norms at r=0.72 (original 0.81); arousal (r=0.67 vs 0.66) is deliberately not certified as replicated because it holds only at full emotion-set scale and late depth.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 0 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What did this replication find about emotion representations in Gemma-2-27B?
Emotion vectors extracted from the base model organize much like the original Claude Sonnet 4.5 study: the top two principal components carry 26.7% and 13.4% of variance and correspond to valence and arousal, emotions cluster into ten intuitive families, and valence correlates with human ratings at r=0.72 (original 0.81). Arousal aligns at r=0.67 (original 0.66) but only at full emotion-set scale and late depth, so the author deliberately does not count it as replicated.
Why is arousal 'not replicated' when its correlation (0.67) matches the original (0.66)?
Because the paper added the criterion that each axis must stay stable across emotion-set sizes and depth regimes, not merely match the original's correlation. Arousal aligns on PC2 only when the PCA uses all 171 emotions and at late layers; refitting on 45 or 27 emotions rotates it off the second component, so it is not a stable, basis-independent feature of the geometry.
Is the paper's data and code available?
The paper states code and data are released, with licensing-aware handling: the 205,200-story Claude-generated corpus ships as a generation recipe rather than redistributed files, gated or non-commercial datasets (LMSYS-Chat-1M, NRC-VAD) ship as provenance plus fetch scripts or hashes, and Gemma-derived tensors ship as regeneration recipes. The 171 vectors, scenarios, and per-token tools are released.
What is the L22–26 'seam' reported in the paper?
In the 46-layer sweep, layers 22–26 are where cross-layer geometric similarity jumps from roughly 0.89 toward 0.999 and emotion words suddenly become readable in the logit-lens output (median own-word rank collapses from about 23,700 at L22 to about 5,400 at L23). A frequency regression (R²≈0.02–0.07) shows the onset is structural, not driven by how common the emotion words are.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text is a sampled excerpt: '[...sampling gap...]' skips include the body of Table 1, portions of §6.1–6.5, §8.2–8.3, §9.1, the §11 reproducibility statement, and part of Appendix B; conclusions about those parts rest only on what is visible elsewhere in the excerpt.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.