The Answer-Basin Representation Hypothesis: We Are Not Probing or Steering Concepts
Manjiang Yu, Hongji Li, Zihan Wang, Junwei Chen, Xue Li, Priyanka Singh, Yang Cao, Lijie Hu
cs.CL
Quick verdict
Claims answer-mass basins, not concept labels, organize LM linear directions; probing/steering evidence is suggestive but correlational; title overclaims.
Summary
The paper proposes the Answer-Basin Representation Hypothesis (ABRH): the linear structure that interpretability methods attribute to concepts is actually organized by the probability measure a language model induces over answers. All continuations yielding the same answer form an 'answer basin'; basin masses define a pushforward measure whose statistics — collision concentration C₂ before generation, pairwise log-mass contrasts after — are claimed to be linearly readable from hidden states via directions shared across questions. Experiments on Qwen2.5-7B/14B and Gemma-3-4B/12B across GSM8K, ARC-Challenge, BBQ and SafetyBench support this: concentration probes reach 63–78% ordering accuracy (Table 1); basin-mass probes reach up to 83.6% ordering accuracy and 86.6% Top-1 on ARC-C (Table 2), and probes trained only on pairs of incorrect answers still recover mass ordering. Reversal experiments show that keeping correctness labels fixed while flipping the label–mass relationship reverses probe predictions (Fig. 3) and steering effects (Fig. 5). Limitations: the geometric evidence is largely qualitative PCA visualization; there is no direct comparison to semantic-entropy or question-only probes; steering was tested on one model and two datasets; finite-choice protocols dominate, and extension to open-ended generation is limited by answer-grouping accuracy.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Hidden states linearly encode statistics of the model's answer measure: question-boundary states order questions by collision concentration C2 (Eq. 5) and answer-boundary states order answers by basin mass (Eq. 6) via directions shared across questions.
- A probe's concept-level behavior is determined by the sign of the label–mass relationship in its training data: flipping that sign while keeping correctness labels fixed reverses held-out concept predictions (Fig. 3; Table 3).
- Basin-mass information is readable without any concept supervision — probes trained only on pairs of incorrect answers recover mass orderings on unseen questions (Table 2).
- A fixed label-defined steering vector amplifies whichever answer has greater initial basin mass, producing opposite concept-level effects under alignment versus conflict (Fig. 5), and source mass gaps shape vector magnitude and directional coherence (Fig. 7).
Strengths
Novel, mathematically careful reframing
Replaces 'concept' with 'statistics of the answer measure' as the organizing object: pushforward measure over answers (Eq. 1), collision concentration C2 = Σ m(a)² (Eq. 2, with unbiased estimator Eq. 9), and a correct Radon–Nikodym change-of-measure identity (Prop. A.1) linking token-level interventions to answer-level masses.
Wrong-only dissociation isolates mass from labels
§3.4/Table 2: probes trained only on pairs of incorrect answers (no concept-label contrast) still recover basin-mass ordering on held-out ARC-C questions (80.00±0.31% ordering for Qwen2.5-7B), directly supporting 'not probing concepts'.
Align/Conflict design cleanly separates label from mass
Steering account connects geometry to behavior
Breadth and protocol transparency
Weaknesses
Geometric evidence is qualitative
No baseline against closest prior work
Narrow steering validation
Surface-form confounds not fully excluded
Statistical reporting gaps
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper engages the Linear Representation Hypothesis (LRH; Park et al., 2024) and its two main applications, probing and steering. It does not dispute linearity; it re-derives *what* the linear directions encode. The formal objects: for a question q, a measurable map φ_q assigns each token continuation an answer identity; fibers ('answer basins') B_q(a) carry total probability m_q(a), defining a pushforward answer measure μ_q (Eq. 1). Two statistics do the explanatory work: collision concentration C2 = Σ m(a)² (Eq. 2, Simpson index), read at the question boundary, and the log-mass contrast Δlog m_q(a,b) (Eq. 3), read at the answer boundary. The Answer-Basin Representation Hypothesis posits shared directions w_Q and w_A realizing strict monotone inequalities (Eq. 5–6), plus a 'transport' prediction for steering (Eq. 8): positive movement along a mass-oriented direction amplifies initially higher-mass answers relatively more. Proposition A.1 correctly formalizes, via a Radon–Nikodym identity, how token-level interventions propagate to answer-level masses. The headline claim — 'we are not probing or steering concepts' — is that concept-consistent behavior of probes and vectors is an artifact of whether external labels align with the endogenous answer measure.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is an 'answer basin' in this paper?
For a given question, an answer basin is the set of all model continuations — sampled generations, or scored candidate options — that yield the same final answer. Its 'mass' is the total probability of all those continuations. The masses across answers form a probability distribution over answer identities (the 'answer measure'), which the paper argues is what actually organizes the linear structure found in the model's hidden states.
What does the title 'We Are Not Probing or Steering Concepts' mean?
The authors argue that when linear probes or steering vectors appear to detect or control concepts like correctness, they are actually reading or amplifying statistics of the model's answer probability distribution (which answer carries more probability mass). Whether the effect looks 'concept-consistent' depends on whether the concept labels happen to line up with the answer masses in the data — flip that alignment and the same probe or vector reverses its apparent concept-level effect.
Why can activation steering backfire, according to this paper?
Because a steering vector built from correct-minus-wrong representation contrasts amplifies the answer that already has more basin mass. When the correct answer has more mass ('alignment'), the intervention improves correctness; when the wrong answer has more mass ('conflict'), the same fixed vector increases preference for the wrong answer. This reversal is demonstrated on Gemma-3-4B with BBQ and SafetyBench (Fig. 5).
Which models and datasets does the paper use?
Qwen2.5-7B/14B (base) and Gemma-3-4B/12B (pretrained) on GSM8K, ARC-Challenge, BBQ, and SafetyBench, plus Qwen2.5-3B-Instruct for a free-form chain-of-thought GSM8K experiment. The authors link code on GitHub, and all datasets and checkpoints are public.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures (PCA scatters, steering curves) are not analyzable from the extracted text; assessments of visual separation rely on captions and the authors' claims.
- The linked GitHub repository's contents were not verified in this review.
- arXiv listing (2609.24821, Sep 2026) indicates a preprint; peer-review status unknown.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.