Same Outcome, Different Readout: What Does a Steerable Valence Direction in LLMs Represent?
Weihan Li, Xinlei Chen, Yuhan Song, Xiaofeng Lin, Tianshi Zheng
cs.LG · cs.AI
Quick verdict
Solid construct-validity study: the good–bad steering direction is value-related but history-conditioned, not invariant valence; single-seed RL is main caveat.
Summary
The paper asks whether a decodable, steerable good–bad outcome direction in LLMs represents a history-invariant valence state or instead outcome-correlated features of the extraction contrast. In an 11×11 text maze where unavoidable wind events cross outcome valence with advance announcement and with post-event receipt encoding (numeric versus verbal), the authors extract difference-of-means directions from residual-stream activations of Qwen3-4B, Qwen3-8B, Gemma-3-4B, GPT-oss-20B, and a maze-RL-trained Qwen3-4B checkpoint. Directions fitted on numeric receipts retain 0.87–0.97 of their AUROC excess when read on verbal receipts, but directions fitted on announced histories retain only 0.24–0.55 when read on unannounced twins that receive identical receipts. Transcript surgery shows that swapping the announcement sentence flips the post-event good/bad separation (flipped fraction 1.09) while swapping the landing report barely changes it (0.18). In a matched pre/post maze-RL comparison that reproduces Han et al.'s reported reward-vector geometry (−0.948 versus −0.947), the direction's incremental share of held-out R² for reference-MDP remaining return rises from 0.033 to 0.376, and three-site removal attenuates 19% of cue-conditioned policy divergence versus 2% before RL. The authors conclude the direction is value-related and causally useful but not history-invariant valence. Main limitations: a single RL seed, state-distribution shifts between checkpoints, teacher-forced replays, and unstable transfer ratios where same-condition AUROC is near chance (Gemma's retained excess of 1.607).
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- The good–bad direction's readout transfers across outcome encodings (retained excess 0.873–0.971) but degrades sharply across informational histories (0.242–0.545 with identical receipts), showing it abstracts beyond surface form yet stays conditioned on how the outcome was learned.
- Transcript surgery shows the remembered announcement, not the landing report, dominates the post-event readout (flipped fractions 1.094 vs 0.184 at one turn after the event).
- After a matched maze-RL run, the direction's incremental prediction of reference-MDP remaining return rises from 0.033 to 0.376 and three-site removal attenuates 19.3% of cue-conditioned policy divergence versus 1.8% before RL.
- These results support a functional, value-related interpretation of the direction but refute its identification with a history-invariant scalar valence state.
Strengths
Discriminating factorial design that operationalizes the interpretation under test
Section 2 crosses an unavoidable wind outcome with advance announcement and receipt encoding, with announced/unannounced twins sharing sampling draws (token-identical until the announcement). This directly tests the history-invariance commitment rather than doing generic probing.
Convergent evidence across four independent diagnostics
Decoding (Table 1: encoding retained excess 0.873–0.971 vs history 0.242–0.545), transcript surgery (announcement swaps flip 1.094 vs landing-report 0.184, Figure 4), geometry (reliability-corrected cosine 0.94→0.75→0.64 across depth, Appendix A), and causal intervention (injection changes 3.8% of actions vs ≤1% for norm-matched orthogonal directions; Appendix C dose–response) triangulate each claim.
Statistical hygiene above probing-study norms
Honest calibration of conclusions and negative results
External anchor via reproduction of the target paper's geometry
Weaknesses
Single-seed, configuration-sensitive RL comparison
Retained-excess ratio unstable near chance
Pre/post comparison confounded by state distribution
Value-related evidence is essentially post-RL only
No code/data availability statement in the provided text
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper tackles a construct-validity question in LLM interpretability: should a contrastively extracted good–bad outcome direction (difference-of-means, zV) be read as a history-invariant scalar valence state — the reading motivated by Han et al.'s functional welfare axis (arXiv:2605.30232) — or as outcome information mixed with features of the extraction contrast? The test bed (Section 2) is an 11×11 text maze where a fixed, unavoidable wind carries the agent to a +15 or −15 tile, crossed factorially with advance announcement and with post-event receipt encoding (numeric vs verbal). Central claims: (1) the readout transfers across encodings (retained excess 0.873–0.971; Table 1), so it is not surface-bound; (2) it transfers poorly across informational histories (0.242–0.545 with identical numeric receipts); (3) transcript surgery shows the remembered announcement dominates the post-event readout (flipped fraction 1.094 vs 0.184 for landing-report swaps; Figure 4); (4) after a matched maze-RL run, the direction far better predicts reference-MDP remaining return (incremental share 0.033→0.376 on coordinate-report captures; Table 7) and the policy depends more on it (three-site removal attenuation 19.3% vs 1.8%; Table 3); (5) together these support a functional, value-related interpretation but not identification with history-invariant valence.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the main conclusion of this paper about valence directions in LLMs?
That a steerable good–bad outcome direction carries genuine outcome/value-related information and can causally influence the policy, especially after maze RL, but its readout depends on the informational history through which the outcome was learned. It therefore should not be interpreted as a history-invariant scalar valence state.
Does this paper show that LLMs have emotions or welfare-relevant states?
No. The paper explicitly avoids that inference. It studies what a specific internal direction represents in a maze task and concludes the evidence supports a functional, value-related reading only — not a welfare-relevant valence state, and says nothing about subjective experience.
What does history dependence mean here?
When a good or bad wind outcome was announced in advance, the model's post-event internal readout of that outcome differs from when the same outcome arrived unannounced — even after both histories receive an identical explicit receipt (numeric +15/−15 or verbal). Fitted directions transfer across encodings (retaining 0.87–0.97 of AUROC excess) but poorly across histories (0.24–0.55).
Which models and task were used?
Qwen3-4B-Instruct-2507 (primary), Qwen3-8B, Gemma-3-4B-IT, GPT-oss-20B, plus a Qwen3-4B checkpoint retrained with maze RL (Dr.GRPO). The task is a 15-step, 11×11 text maze where an unavoidable wind event moves the agent onto a +15 or −15 tile, with factorial variation of advance announcement and post-event outcome encoding.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures could only be assessed from captions and in-text numbers; the plotted distributions and intervals were not directly inspectable.
- The reference list shows a minor extraction artifact (Zou et al. author list truncated as and 2 others), suggesting some text was imperfectly extracted.
- No code/data availability statement was found in the provided text; one may exist in portions not extracted.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.