How do LLMs Evaluate Perceived Moral Agency? Investigating Moral Decision-Making in Human-Artificial Agents Interactions
Fernanda Mansilla, Aloysius Tok, Bahia Guellaï, Farah Benamara, Nancy F. Chen
cs.CL · cs.AI
Quick verdict
First human-vs-LLM study of perceived moral agency in situated dilemmas; rigorous protocol, but conclusions rest on 4 small models chosen for human alignment.
Summary
This paper reports what the authors describe as the first empirical comparison of how humans and LLMs attribute Perceived Moral Agency (PMA). Extending Banks's (2019) validated PMA scale, the authors add eight smart-city scenarios spanning three situations — AAs offering help, witnessing violations, or needing help — scored on Autonomy, Action Endorsement, and Moral Judgment. 190 human participants completed a between-subjects survey (one agent each); twelve LLMs were surveyed through a three-phase protocol (SAE-based selection, consistency at near-deterministic temperature, prompt sensitivity via PSS), retaining Phi-4-14B, Llama-3.1-8B, Llama-3.2-11B, and multimodal InternVL3-38B. Both evaluator types rate humans above artificial agents on PMA; LLMs inflate situated ratings of AAs beyond their dispositional scores, overestimate AA autonomy relative to human raters, and uniformly endorse AAs assisting humans. A qualitative analysis of 222 model explanations (46 codes, two raters) shows identical scores can arise from different reasoning, with harm prevention dominant. Main limitations: the LLM generalization rests on four small models selected for human alignment (Gemini excluded from the quantitative arm by high SAE), qualitative autonomy reliability is weak (r=0.40), the human sample skews female, and code/data are only promised for release upon publication.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Humans and LLMs both attribute higher perceived moral agency to humans than to autonomous artificial agents, replicating Banks (2019) with a new evaluator type.
- LLMs rate AAs' situated moral judgment above what their dispositional PMA scores predict, amplifying a context-sensitivity already present in human raters.
- LLMs overestimate the autonomy of artificial agents relative to human raters, in tension with their own dependency-style assessments.
- Identical LLM scores can arise from substantially different justificatory strategies, so explanation analysis is a necessary complement to numeric scale scores.
- Multimodal (image+text) prompting consistently degrades alignment with human moral-agency ratings relative to text-only prompting.
Strengths
Reliability-first LLM survey protocol
Section 3.3's three phases — SAE-based selection (Table 5), 3-run consistency (Phi-4-14B CR=0.976, κ=0.978, α=0.991; t=0.3 rejected with CR dropping to 0.22–0.71), and prompt-sensitivity with a convergent validity check (all PSS <0.026; PSS-ProSA Spearman ρ=0.954, n=132) — treat reliability as an explicit selection criterion rather than a post-hoc check.
Psychometrically validated new instrument
8 scenarios × 9 items across three situations, content-validated by a 4-expert panel, Cronbach's α 0.75–0.95 with the weak cell (moral judgment, situation b: 0.627 [0.539, 0.704]) reported with a CI; gender confound checked via White-corrected Type II ANOVA (no Gender × Agent interaction; one marginal Dependency effect, F=5.013, p=.026).
Dispositional + situational design exposes what scale-only evaluation hides
Honest mixed-methods reporting
Negative results reported plainly
Weaknesses
Selection circularity constrains the headline generalization
Weak qualitative reliability under the autonomy discussion
SAE alignment metric lacks normalization and uncertainty
Sample breadth limits
Artifacts promised, not released
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This JAIR-format preprint (arXiv:2609.05037, cs.CL/cs.AI) asks how LLMs evaluate Perceived Moral Agency (PMA) — the agency observers attribute to an agent regardless of whether it actually has it (Banks 2019). Claimed contributions: (1) an instrument extending Banks's scale with 8 smart-city scenarios (9 items each) across Autonomy, Action Endorsement, and Moral Judgment; (2) a three-phase protocol for surveying LLMs; (3) evidence that LLMs' situated ratings of artificial agents (AAs) exceed what their dispositional PMA scores predict; (4) a qualitative, code-based analysis of explanations. Headline claims: humans and LLMs both rate humans above AAs; LLMs amplify a context-sensitivity already present in humans; LLMs overestimate AA autonomy; identical scores hide different reasoning; multimodal input degrades alignment.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Perceived Moral Agency (PMA)?
PMA is the extent to which an observer attributes moral capacities — sensing right and wrong, feeling obligated, acting on moral reasons — to an agent, regardless of whether the agent actually has them. This paper uses Banks's (2019) 10-item PMA scale (morality + dependency) and extends it with 8 smart-city scenarios measuring autonomy, action endorsement and moral judgment.
Which LLMs best matched human moral-agency ratings?
Small open models did best: Phi-4-14B (SAE 22.29) and Llama-3.1-8B-Instruct (22.48) beat Llama-3.3-70B (38.41) and Gemini-2.5-Pro (56.08). Adding images consistently worsened alignment. The four retained models were Phi-4-14B, Llama-3.1-8B, Llama-3.2-11B (text-only) and InternVL3-38B (image+text).
Do LLMs see robots or AI as moral agents like humans do?
In this study, both humans and the selected LLMs attribute more moral agency to a human than to a drone, robot or disembodied AI. But LLMs raise their ratings of the AI agents in concrete dilemma scenarios, overestimate the agents' autonomy compared with human raters, and almost always endorse AI helping humans.
Is the paper's code or data available?
The reproducibility checklist states that all code, data, prompts and model outputs will be released in a GitHub repository upon publication; per-model error tables, metric formulas and example prompts are already in the paper's appendices.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided excerpt is sampled with marked gaps: participant recruitment/demographics (Sections 3.1–3.2), the main quantitative results (Section 4), the latter part of the Discussion, the Limitations section, the Conclusion, and parts of the appendices were skipped; this review makes no judgment about content it could not see.
- No effect sizes appear anywhere in the provided text; if the paper reports them, they would be in the unsampled Section 4.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.