Contrastive Projection: Reading Transformer Internals by Differencing Logit Lenses
cs.CL
Quick verdict
Reads transformer internals by differencing matched prompts through the logit lens; compelling, honest case studies but small-N, largely qualitative evidence.
Summary
The paper turns a known primitive—subtracting the hidden states of two closely matched prompts and projecting the difference through the unembedding (equivalently, reading a RepE/ActAdd steering vector through a logit lens)—into a systematic, training-free tracer. Contributions include per-position, sub-layer, and per-head readouts; multi-contrast triangulation that averages over designed baselines to cancel incidental axes; and contrast-design rules (shared preamble, matched current token, matched predicted next token). Case studies in Phi-2 trace a compound-noun MLP→attention chain (hot dog vs cold dog) confirmed by activation patching, and decompose it into edibility and petness axes that are jointly causal. The distinction is recovered in Pythia-1.4B and Qwen2.5-1.5B via triangulation plus linear probes. Real versus fictional entity contrasts read factual associations on real poles and only name fragments on fictional poles, robust to a length-matched control. Metaphor reads as domain-to-domain routing directions, not a figurativity flag. A cross-seed control over five Pythia-410M models shows top-10 token overlap of only 0.08 while the distinction holds in every seed. Limitations, largely self-acknowledged: hand-curated minimal pairs selected from several hundred readings, coherence judged by inspection, qualitative patching without self-repair controls, and a key visibility threshold deferred to a companion preprint.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Subtracting two closely matched prompts' hidden states before WU projection cancels shared content and surfaces the distinguishing content as legible tokens, equivalent to reading a RepE/ActAdd steering vector through a logit lens.
- In Phi-2, the hot-dog/cold-dog distinction is written by an MLP at L4 at the noun, routed by head L5.H19 to the prediction site, with activation patching confirming the two-hop chain and edibility+petness acting as a jointly causal plane.
- The compound-noun distinction is decodable in Phi-2, Pythia-1.4B, and Qwen2.5-1.5B, but a token face at the noun is legible only in Phi-2 (and partly Pythia via triangulation), making single-pair illegibility an instrument effect rather than an absence.
- Real-entity recall surfaces factual associations while fictional-entity hallucination surfaces only name fragments and generic number priors, robust to token-length matching; entropy alone cannot separate the two cases.
- Token identities are network-specific (mean top-10 Jaccard 0.08 across five Pythia-410M seeds) while the underlying distinction holds in every seed and contrast (20/20 margins), so the axis, not the tokens, should be read.
Strengths
Design rules are measured, not asserted
§2.5 grounds each contrast-design rule in data: shared preamble justified by the first-token massive activation (norm 941 vs ~45 at L8); current-token mismatch shown to inject suffix morphology with norm 30 vs 3–5 at L1 (Table 5); capitalization mismatch rotates the read from cosine 0.85 to 0.64 (Table 2).
Observation is anchored to causal checks where causal claims are made
The Phi-2 compound-noun chain (MLP@L4 norm 20.0 vs attention 3.8; head L5.H19 attending 0.911 vs 0.735) is confirmed by activation patching at three positions (Table 14: patching 'dog' at L2 drops P(food) to 0.00; patching 'was' at L20 to 0.00), and axis injection shows edibility+petness jointly recover 0.98 vs 1.00 while each alone manages ≤0.03 (Table 12).
The cross-seed control is a durable, reusable finding
Anticipates and controls the obvious confounds
Code and data released
Weaknesses
Severe selection exposure on legibility claims
The quantitative legibility criterion lives in a companion preprint
Perfect probe accuracy on small curated sets
Patching is qualitative by the authors' own account
Thin behavioral confirmation outside Phi-2
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper (arXiv 2609.09902, cs.CL; single author, Evident Solutions Oy) proposes a training-free interpretability tracer: subtract the hidden states of two closely matched prompts and project the difference through the unembedding — explicitly framed as reading a RepE/ActAdd steering vector through a logit lens. The primitive itself is credited to Du et al. (2026); the claimed contributions are systematization: per-position, per-sub-layer, and per-head trajectory reading; multi-contrast triangulation over designed baselines; and contrast-design rules (§2.5). Applications span a compound-noun circuit in Phi-2, cross-model recovery in Pythia-1.4B and Qwen2.5-1.5B, real-vs-fictional recall, metaphor as domain routing, and a cross-seed control on five Pythia-410M models. Claims are consistently scoped: the abstract's closing thesis — token appearance is network-specific (top-10 overlap 0.08) while the distinction is not — is the paper's best-supported statement.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is contrastive projection in transformer interpretability?
It is the operation of subtracting the hidden states of two closely matched prompts and projecting the difference through the model's unembedding matrix (a logit lens read of the difference). Content the two prompts share cancels, so the top tokens of the projection name what separates them. Mathematically it is the same as reading a RepE or ActAdd steering vector through a logit lens.
Is reading an activation difference through the logit lens new?
The primitive is not: the paper credits Du et al. (2026) with decoding an activation difference through the logit lens, and the operation is the arithmetic of RepE/ActAdd steering vectors. The claimed novelty is systematization — per-position, per-head, and sub-layer readouts, averaged multi-contrast baselines, and contrast-design rules — used read-only rather than for steering.
Do the same tokens appear when you repeat the analysis on different model seeds?
No. Across five Pythia-410M models differing only in initialization seed, the mean pairwise top-10 token overlap was a Jaccard of 0.08 (about 45 distinct tokens filling 50 slots), yet the underlying distinction was detectable in every seed and contrast (20/20 cases). The paper concludes token identities are network-specific while the distinction is shared — hence 'read the axis, not the tokens'.
Can this method detect hallucinations?
Not directly, and it finds no 'hallucination flag'. For fictional entities the readout surfaces only name fragments and generic number priors, while real entities surface factual associations — a diagnostic of what retrieval emits. Entropy alone fails to separate the cases (2.14 for a real mountain vs 2.36 for a fictional one at near-identical confidence), whereas the projection does.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- PDF extraction renders multi-column tables (e.g. Tables 12 and 34) as inline text; cell values are verbatim but column grouping was reconstructed from the surrounding prose. The extracted text runs complete through the references page (27) with no truncation.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.