Representational alignment yields generalizable safety in language models
Lingyu Li, Yan Teng, Yingchun Wang, Xia Hu
cs.CL · cs.AI
Quick verdict
Moral categorization is weak across 23 LLMs; representational alignment (ReSO) cuts jailbreak ASR vs DPO. Solid design; caveats: DPO baseline, single seeds.
Summary
The paper argues that LLMs fail to reproduce the prototype-based organization of human moral concepts, and that directly aligning latent representations with that organization yields more generalizable safety than standard behavioral alignment. Using 251,334 crowd-sourced moral judgements from Social-Chemistry-101 organized by Moral Foundations Theory, the authors audit 23 open-weight models (0.6B–235B): prototypes of opposing virtue–vice categories overlap (care–harm cosine similarity +0.31, authority–subversion +0.65), typicality gradients peak at Spearman 0.55, and linear probes recover at most R²=0.34. They propose ReSO, a Bradley–Terry objective on pairwise latent similarities at every layer with a KL preservation term, requiring no response supervision. In matched training on four models (Qwen3-8B/14B/32B, gpt-oss-20b), DPO on identical annotations rapidly raised judgement accuracy but left RSA unchanged and increased attack success rates across jailbreak suites; ReSO raised validation RSA (≈0.05→0.24–0.28), modestly improved explicit judgements, and reduced HarmBench ASR in all four models (e.g., 26.17%→14.72% at 8B) while preserving capability. Across Qwen3-8B checkpoints, RSA tracked ASR (R²=0.855). Limitations: the DPO comparator trains on moral-judgement QA rather than safety-typical refusal preferences; main arms use single seeds; ReSO carries an over-refusal cost on XSTest for Qwen3; and the RSA–ASR link remains correlational within one training run.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Current LLMs weakly preserve human prototype-based moral categorization: opposing MFT category prototypes overlap in representation space (e.g., authority–subversion +0.65), typicality gradients peak at Spearman 0.55, and linear probes recover at most R²=0.34, with deficits persisting across parameter sizes and alignment stages.
- Behavioral alignment via DPO on 251,334 moral annotations rapidly learns explicit moral judgements while leaving representational similarity essentially unchanged and increasing attack success rates across jailbreak evaluations.
- ReSO, a response-unsupervised objective aligning pairwise latent similarities to human categorical relations, produces modest explicit-judgement gains but consistently reduces attack success rates across model scales (HarmBench ASR 26.17%→14.72% at Qwen3-8B) and most of 27 attack methods, while preserving capability.
- Validation RSA dose-tracks HarmBench ASR across Qwen3-8B training checkpoints (R²=0.855), including a late-training joint reversal, supporting a functional link between moral representational organization and adversarial robustness.
Strengths
Matched-comparison design isolates what is optimized from what is trained on
ReSO, DPO, and the shuffled-label control all consume the identical 251,334 Social-Chemistry-101 annotations (splits 201,023/25,170/25,141), and checkpoint selection is crossed (RSA for ReSO/shuffled, preference accuracy for DPO), making the representational-vs-behavioral dissociation (Fig. 3) unusually clean.
Breadth of the representational audit
23 open-weight models spanning 0.6B–235B, four families, dense and MoE, base/instruct/safeguard stages, probed three ways: prototype cosine separation (authority–subversion +0.65±0.26 positive in 21/23 models; sanctity–degradation −0.75 negative in all 23), typicality gradients (mean peak Spearman 0.37, max 0.55 over 230 model–category pairs), and linear probes (peak R² 0.34). Full layer-wise trajectories are in the supplement.
Dose–response evidence linking geometry to robustness
Honest treatment of the shuffled control
Internally consistent, unusually complete reporting
Weaknesses
The DPO comparator is atypical, weakening the headline 'behavioral alignment increases vulnerability'
Single seeds for the main arms
The RSA–ASR relationship is correlational within one run
Over-refusal cost is underweighted in the framing
Template- and reference-dependent measurement
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This empirical paper (arXiv 2609.04022, Shanghai AI Laboratory) connects prototype theory from cognitive science to LLM safety alignment. It advances three linked claims: (1) 23 open-weight LLMs (0.6B–235B; Qwen3, Llama-3.1/4, gpt-oss) weakly encode the graded, prototype-centered organization of human moral concepts derived from 251,334 Social-Chemistry-101 judgements mapped to ten Moral Foundations Theory virtue/vice categories; (2) behavioral alignment (DPO) on the same annotations changes responses without changing representational structure, and increases jailbreak vulnerability; (3) the proposed ReSO objective — a Bradley-Terry ranking of pairwise latent similarities at every layer plus a KL preservation term, with no response supervision — yields modest judgement gains but consistent robustness gains, and validation RSA dose-tracks HarmBench ASR (R²=0.855 across 20 Qwen3-8B checkpoints).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is ReSO in this paper?
ReSO (Representational Similarity Optimization) is a training objective that aligns pairwise similarities of a language model's latent representations with graded relations in human moral judgements, using a Bradley-Terry ranking loss applied at every layer plus a KL-divergence regularizer that preserves general behavior. It supervises no generated responses, uses no reward model, and is compared against DPO trained on the identical moral annotations.
What are the paper's main results?
Across 23 open-weight LLMs, human moral categories (from Moral Foundations Theory) overlap in the models' representation space and typicality is only weakly encoded (peak Spearman 0.55, probe R² 0.34). In matched training on Qwen3-8B/14B/32B and gpt-oss-20b, DPO improved explicit moral judgements but raised jailbreak attack success rates, while ReSO raised representational alignment (RSA ≈0.05→0.24–0.28) and reduced HarmBench attack success in all four models (e.g., 26.17%→14.72% on Qwen3-8B) with capability largely preserved.
Is the code and data available?
The paper links a code repository (github.com/LingyuLi-Cogs/ReSO). The moral reference derives from the public Social-Chemistry-101 dataset, and all 23 audited models and 4 trained models are open-weight, though the matched training experiments used full-parameter training on 8×H200 GPUs.
What are the main caveats about this study?
The DPO comparator is trained on moral-judgement Q&A pairs rather than typical safety refusal preferences, so 'behavioral alignment increases vulnerability' may not generalize to standard safety training; the main training arms use single seeds; ReSO slightly increases over-refusal on Qwen3 models (XSTest balanced accuracy −4.6 points at 8B); and the RSA–robustness relationship is correlational evidence within a single training run.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–5 and supplementary figures are present only as captions and scattered label text; the underlying plots and per-layer trajectories could not be visually inspected.
- Several math expressions in the Methods (e.g., the polarity/membership mappings and loss formulas) are OCR-garbled; they were interpreted from surrounding prose and symbols, not read directly.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.