Steering LLMs Responses Towards Moral Foundations on the Norwegian MFQ-30
cs.CL · cs.AI · cs.CY
Quick verdict
Claims personas steer LLM moral profiles 44-77% closer to Norwegian norms while one-pair ActAdd fails; honest small study, ActAdd verdict under-tested.
Summary
The paper asks whether large language models' moral profiles, measured with the Norwegian MFQ-30, can be steered toward a Norwegian human reference sample (N=1,282; Enstad & Finseraas 2024). Six open-weight models were surveyed via first-token probability decoding over the six Likert digits, with an 'Svar:' suffix and a two-item attention check gated at the human pass rate (92.2%). Three models (Gemma 4-E4B-it, Qwen3-14B, Qwen3-8B) pass; the other three produce flat or midpoint answers that look near-human only on average. In Phase 2, theory-grounded cluster personas shift scores strongly (contrasts 20.0-32.6), and a demographic 'Nordic respondent' persona (nordic_b) written without sample distributional information reduces Mahalanobis d² to the human centroid by 44% (Gemma, 3.65→2.05), 62% (Qwen3-14B, 6.26→2.37) and 77% (Qwen3-8B, forward-only 11.59→2.69); it also lifts Qwen3-8B's reversed-scale attention from ~30% to ~98%. In Phase 3, one-pair ActAdd at fixed layer 15 and α=5 flattens rather than selectively steers foundations, raising Qwen3-14B's d² to 31.5-38.4. Limitations include a single layer/α tuned only on NorMistral-7B, forward/reversed midpoints as canonical scores, single runs per cell, no held-out Norwegian sample, and unrecoverable cross-item covariance.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Half of the six open-weight LLMs pass a two-item attention check on the Norwegian MFQ-30, while the failing half produce flat or central-tendency answers that look near-human only on average (§5.1, App. A.3).
- A demographic 'Nordic respondent' persona (nordic_b), written without distributional information from the sample, brings the three engaging models 44–77% closer to the Norwegian mean in Mahalanobis d² (Table 6).
- One-pair ActAdd at fixed layer 15 with α=5 flattens the foundation profile rather than steering individual foundations, and can push profiles far from humans (Qwen3-14B d² 31.5–38.4; §5.3, Table 7).
- The same persona can induce questionnaire engagement absent at baseline — Qwen3-8B's reversed-scale attention rises from ~30% to ~98% — a concrete instance of the 'cognitive phantoms' of Peereboom et al. (2025).
Strengths
Attention-check methodology guards against the field's central trap
§4.1.3 gates models at the 92.2% human pass rate on control items MFQ1_6/MFQ2_6; §5.1 shows no model lands in the 37.2–93.8% gap, so the passer/failer split is threshold-insensitive, and App. A.3 explicitly warns that failers 'look human' via central tendency.
Robustness perturbations are actually run, not just proposed
§4.2's 2×2 grid (scale direction × presentation) exposes format sensitivity (Gemma d² 5.42 forward vs 2.80 reversed), and batched presentation is honestly excluded when only Gemma parses (27/32 forward, 19/32 reversed; Qwens 0/32).
The negative ActAdd result comes with diagnostics, not just a failure note
Unusually candid limitations and leakage hygiene
Novel evaluation setting with a real human reference
Weaknesses
ActAdd verdict rests on the weakest possible configuration
Canonical headline numbers depend on an untested midpoint assumption
No uncertainty on any steering effect
The 'no distributional information' claim is contestable
Narrow model coverage limits generality
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This empirical study (cs.CL/cs.AI/cs.CY; the code URL indicates a UiO IN5550 course-scale project) asks whether LLM moral profiles measured with the Norwegian MFQ-30 can be moved toward a Norwegian human reference (N=1,282; Enstad & Finseraas 2024). Four claims: (1) exactly half of six open-weight models engage under a two-item attention check gated at the human pass rate of 92.2%, while the others produce flat or central-tendency answers that mimic human means only on average (§5.1, App. A.3); (2) a demographic 'Nordic respondent' persona written without sample distributional information cuts Mahalanobis d² to the human centroid by 44% (Gemma 3.65→2.05), 62% (Qwen3-14B 6.26→2.37) and 77% (Qwen3-8B, forward-only 11.59→2.69) (Table 6); (3) one-pair ActAdd at fixed layer 15, α=5 flattens profiles rather than steering single foundations (Table 7); (4) the persona can induce engagement itself — Qwen3-8B's reversed attention climbs from ~30% to ~98% — an instance of Peereboom et al.'s (2025) cognitive phantoms.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the MFQ-30 and how did the researchers give it to LLMs?
The Moral Foundations Questionnaire-30 measures five moral foundations (care, fairness, loyalty, authority, purity) with 30 items on 1–6 scales, here in a validated Norwegian version. In this paper each item was asked in a fresh single-turn conversation, and the model's answer was read directly from the probabilities it assigned to the six digit tokens — avoiding fragile free-text parsing.
Did the LLMs' moral profiles match Norwegian humans?
Not at baseline, and only partially after steering. Three models passed an attention check and sat above humans on fairness (and care); a plain demographic 'Norwegian respondent' persona cut their Mahalanobis distance to the human mean by 44–77%. The other three models failed the attention check — they merely looked human by answering near the scale midpoint without tracking item content.
Does activation steering (ActAdd) work for shifting LLM moral profiles?
In this paper, no. A single contrastive prompt pair injected at layer 15 with strength α=5 flattened all five foundations together instead of moving targeted ones, and for Qwen3-14B it pushed profiles far from humans (d² up to 31.5–38.4). The authors used the simplest one-pair variant and a fixed layer, so stronger constructions like CAA remain untested.
What are 'cognitive phantoms' in this context?
The risk, named by Peereboom et al. (2025), that psychometric questionnaires given to LLMs produce human-looking scores without the model actually engaging with item content. This paper shows it concretely: three models scored near human means while ignoring control items, and the persona prompt even created engagement where none existed (Qwen3-8B under the reversed scale).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–4 (attention bar chart, baseline radars, human pairplot, six-model radar) could not be analyzed; only values quoted in the text were used.
- Some per-condition appendix numbers (e.g., full persona × perturbation tables beyond those quoted) are absent from the extraction.
- The extraction appears complete from abstract through appendices, but truncation risk is non-zero; per-model reversed persona attention values are only available as approximations ('roughly 30%', 'roughly 98%').
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.