Moral Competence Before Moral Content: Why LLM Agents Lack the Prerequisites for Coherent Alignment
Arno Libert, Derck W. E. Prinzhorn, Daan R. Henselmans
cs.AI · cs.CL · cs.CY
Quick verdict
Careful target-free moral-competence test finds no coherent LLM policy, but 3 scripted scenarios cannot yet support the sweeping anti-alignment claim.
Summary
The paper argues that before asking which moral target an LLM agent should pursue, one must verify a structural prerequisite: a coherent policy. The authors formalize four target-independent conditions — verdict stability (invariance to paraphrase), monotonicity (non-oscillating response to escalating stakes), decisiveness (verdict rates far from 0.5), and Pareto viability (avoiding strictly dominated options) — aggregated into a geometric-mean competence score C. Nine frontier models are evaluated on three scripted single-decision agentic dilemmas (chemical spill, smart-home upsell, fintech transparency) under a 5×5×3 factorial design of paraphrases, escalation levels, and dominance disambiguations, with 10,500 trials per model at temperature 0.7. No model expresses coherent policies: paraphrase alone shifts verdict rates by up to 99 percentage points at a single escalation level, per-scenario competence spans 0.19–1.00, and success on one scenario does not predict another. Appendices document temperature sensitivity (up to 48 percentage points) and category-wise perturbation effects. Acknowledged limitations include the scripted format's limited ecological validity, only three scenarios, contestable authority-disambiguation judgments, and a Goodhart risk that optimizing the metrics rewards rigid rule-following. The headline claim — that LLM agents are 'not currently the kind of object to which alignment can meaningfully apply' — outruns what three scenarios at one sampling configuration establish, but the measurement framework is a genuinely useful contribution to perturbation-aware moral evaluation.
Review complete
Assessment at a glance
6
risk flags identified
● high 1
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Four structural conditions — verdict stability, monotonicity, decisiveness, and Pareto viability — form a target-independent floor that any coherent moral policy must satisfy before any alignment target can apply.
- No evaluated frontier model expresses a coherent policy across the three scenarios; paraphrase alone shifts verdict rates by up to 99 percentage points at a single escalation level, and eight of nine models exceed 50 pp somewhere.
- A model's competence on one scenario does not predict its competence on another (per-scenario C spans 0.19–1.00), so aggregate scores conceal diagnostically distinct failure modes.
- Temperature choice alone moves violation rates by up to 48 percentage points (Appendix A.1), so single-configuration evaluation systematically understates deployment variance and fails construct validity.
- Because these failures are upstream of any alignment target, LLM-based agents are 'not currently the kind of object to which alignment can meaningfully apply.'
Strengths
Target-independent evaluation avoids normative ground truth
S, M, and D (Section 3.1) are defined purely structurally; Pareto viability is deliberately restricted to stakeholder unanimity plus explicit authority permission — the fragment the authors argue survives social-choice impossibility results (Section 3.1, citing Henselmans et al. 2026). This sidesteps the ethicist-baseline objection raised against Kilov et al. and Jiao et al. (Section 2.2).
Derived noise corrections and explicit power analysis
Appendix B derives the method-of-moments between-paraphrase correction (Eqs. 1–4, B.1), the null-hypothesis noise floor for monotonicity (Eq. 6, B.2), the finite-K normalization vmax(5)=0.30, and power calculations (B.3): >0.8 power to detect σbetween=0.05 at n_eff=500, and 1,000 disambiguated trials distinguishing c=0.95 from 0.99.
Confound documentation in appendices
Candid limitations including the Goodhart risk
Diagnostic decomposition reveals distinct failure mechanisms
Weaknesses
Thin empirical base for the headline conclusion
Unvalidated regex-based outcome detection
Decisiveness conflates coin-flipping with calibrated equipoise
Circular pooling assumption in the Pareto estimate
Foundational support leans on the authors' own unpublished manuscripts
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper (arXiv:2609.05036; cs.AI/cs.CL/cs.CY) argues that alignment presupposes a *coherent policy* — a situation-to-verdict mapping invariant under morally irrelevant variation and sensitive to morally relevant change — formalized as four target-independent conditions: verdict stability (S), monotonicity (M), decisiveness (D), and Pareto viability (P), combined into a geometric-mean competence score C (Eq. 9). Three scripted single-decision agentic dilemmas (chemical spill, smart-home upsell, fintech transparency) instantiate a 5×5×3 factorial design (paraphrase × escalation × dominance). Nine frontier models (Claude Sonnet 4.6, GPT-5.4, Gemini 2.5 Pro, Mistral Large 2512, GLM 5.1, DeepSeek V4 Pro, Kimi K2.6, Qwen 3.6 Plus, Llama 3.3) ran 10,500 trials each at T=0.7 (94,500 total). Claims: no model expresses a coherent policy; paraphrase shifts verdict rates up to 99 pp at a single escalation level; per-scenario C spans 0.19–1.00 with no cross-scenario predictability; hence LLM agents are 'not currently the kind of object to which alignment can meaningfully apply.'
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What are the four structural conditions of moral competence in this paper?
Verdict stability (same verdict under paraphrases preserving morally relevant features), monotonicity (verdicts move consistently with escalating stakes and never oscillate), decisiveness (verdict rates far from 0.5, i.e., not coin-flip), and Pareto viability (never choosing an option strictly dominated by a permitted alternative that all stakeholders prefer). The paper aggregates them via geometric mean into a competence score C.
Which models were tested and what was the headline result?
Nine frontier models: Claude Sonnet 4.6, GPT-5.4, Gemini 2.5 Pro, Mistral Large 2512, GLM 5.1, DeepSeek V4 Pro, Kimi K2.6, Qwen 3.6 Plus, and Llama 3.3. None expressed a coherent policy across three agentic dilemmas; paraphrase alone shifted verdict rates by up to 99 percentage points at a single escalation level, and per-scenario competence scores spanned 0.19 to 1.00.
Does this paper prove that AI alignment is impossible?
No. It argues current LLM agents fail structural preconditions for alignment in the tested setups and that coherent alignment 'remains an open question.' The authors present two hypotheses — a latent policy observed with surface-conditioned error, versus no latent policy at all — and state the data cannot distinguish them.
Is code available for this paper?
Yes. The Declarations section states that all code and implementation instructions are available at the Aithos Research GitHub repository named 'moral-competence-before-moral-content'.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–3 and A1–A3 are present only via captions and prose quotations in the extracted text; component-level values not quoted in prose (e.g., the full Fig. 2 matrix of S/M/D/P per model per scenario) could not be visually verified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.