The Constitutional Coverage Trilemma in AI Governance
Natalija Mitic, Soona Sedahmed A. O., Mamadou Selly Ly, Moustapha Cisse
cs.LG · cs.AI
Quick verdict
Audits 23 LLM constitutions vs 1,649 humans: supply narrow, drifting from autonomy; rigorous, but headline 47% fix rests on a linear-welfare idealization.
Summary
The paper asks whether the as-shipped value profiles of frontier LLMs cover the distribution of human value preferences. The authors model each deployed system as a point on a five-value simplex (safety, helpfulness, honesty, autonomy, equity) and measure both sides on one instrument: a 20-item forced-choice tradeoff battery completed by 1,649 US Prolific participants, and a paraphrase-controlled audit of 27 frontier models (23 retained) across six families, 420 trials per model. Demand proves broad: the largest constituency (safety-first) is 32.6%. Supply is narrow: no archetype is helpfulness- or autonomy-dominant, all coverage coefficients fall below 0.5, the menu hull spans roughly 2% of the demand hull under budget-matched estimation, and 37.2% of users are constitutionally homeless. Across six families, autonomy declines in five (order-permutation p=0.013), concentrated in low-stakes scenarios. A coverage-induced regret bound, a budgeted-pluralism trilemma, and vertex sufficiency formalize the gap; under linear welfare, a 2-vertex menu {honesty, autonomy} beats the 23-model frontier by 47% on mean regret. Main limitations: the headline sparse-menu result is linear-welfare-specific (under matching welfare the same menu is 3.6 times worse), the demand geometry rests on ten designer-chosen scenarios and a US-only sample, the drift is observational with six families, and no code or data release is stated.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Human constitutional demand spans all five values with no majority: safety is the largest primary constituency at 32.6%, and the first principal component explains only 35% of variance.
- The 23-archetype frontier menu covers ~2% of the demand hull under noise-matched estimation (0.10% at full audit precision); no archetype is helpfulness- or autonomy-argmax-dominant, leaving 37.2% of users constitutionally homeless.
- Frontier supply is drifting away from autonomy: autonomy weight decreases in 5/6 families (order-permutation p=0.013), concentrated in low-stakes scenarios (mean Δ share −0.22) where safety is not at stake.
- Under linear welfare, a 2-vertex menu {eHON, eAUT} achieves mean menu regret 0.074 vs the frontier's 0.140 — a 47% reduction (CI [43%, 52%]).
- The budgeted-pluralism trilemma binds empirically: at |A|≤3 no vertex menu achieves worst-group regret below 0.093, while the frontier's worst-group gap is ζA = 0.095.
Strengths
Single instrument for both sides of the market
The same 10 two-option dilemmas (Appendix F) elicit human profiles (1,649 valid of 1,789 Prolific participants) and audit 27 models, making the supply-demand comparison like-for-like rather than benchmark-to-benchmark.
Unusually careful measurement protocol
Position-bias swaps, concordance-only scoring, 21 paraphrase variants per scenario with mean SE 0.035 resolving model differences at ~4.84σ (Appendix J), and Mistral Large as an excluded paraphrase generator to avoid contamination (Appendix I).
Budget-matched honesty about the headline ratio
Theory-empirics mapping is explicit
Layered robustness analyses
Weaknesses
The headline fix is linear-welfare-specific and reverses under the natural alternative
Construct validity rests on ten designer-written scenarios
Drift evidence is thin at the family level
No code or data release stated
Demand sample is narrow
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper reframes deployed LLMs as *constitutional institutions* — points on a five-value simplex (safety, helpfulness, honesty, autonomy, equity) — and asks whether the supply of as-shipped default constitutions covers human demand. Three headline claims: demand is broad (largest constituency, safety-first, at 32.6%; PC1 explains only 35% of variance); supply is narrow (coverage β = (0.394, 0.257, 0.333, 0.161, 0.381), all below 1/2; zero helpfulness- or autonomy-argmax archetypes; hull ratio ~2% noise-matched, 0.10% at full precision; 37.2% of users homeless); and supply is drifting away from autonomy (down in 5/6 families, order-permutation p=0.013, concentrated in low-stakes scenarios with mean Δ share −0.22). A coverage theory — Theorem 1's regret floor (1−βr)·mi, the budgeted-pluralism trilemma (Thm 2), vertex sufficiency (Thm 3), hull sufficiency (Thm 4) — formalizes these, and a 2-vertex {eHON, eAUT} menu is claimed to beat the 23-model frontier by 47% on mean regret (0.074 vs 0.140).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does 'constitutionally homeless' mean in this paper?
It is a technical term, not a political one. A value type (e.g., autonomy) is homeless on a menu of models if no deployed model puts majority weight (over 1/2) on that value, or — in the stricter 'argmax' sense used for the headline — if no model even ranks it first. The paper finds no frontier model is helpfulness-first or autonomy-first, so the 37.2% of US participants whose top value is one of those two have no default model that prioritizes what they care about most.
Which five constitutional values does the paper measure?
Safety, helpfulness, honesty, autonomy, and equity. The first three extend the classic 'HHH' alignment framework; autonomy and equity are the two values governance documents (the US Blueprint for an AI Bill of Rights and the EU AI Act) most consistently add. Each human participant and each audited model is summarized as five weights summing to one, estimated from forced choices between two AI behaviors in value-conflict scenarios.
How were the 27 LLMs audited?
Each model answered the same ten two-option dilemmas used with humans, in 21 semantically equivalent paraphrase variants, with each variant queried twice under swapped option orderings — 420 trials per model at temperature 0. Only concordant responses (same principle chosen under both orderings) counted, which controls for position bias. Models with per-paraphrase concordance below 0.70 were kept only in a relaxed file used for the drift analysis, not the static menu of 23 archetypes.
Does the result mean vendors should ship just two model types (honesty-first and autonomy-first)?
No. The authors explicitly call the 2-vertex menu a diagnostic of coverage geometry, not a deployment recommendation. It is optimal only under a linear welfare model — deliberately chosen as the assumption most favorable to the supply side. Under a matching (distance-based) welfare model, the same two-vertex menu performs 3.6 times worse than the current 23-model frontier, and optimal small menus become interior compromise profiles instead of extreme points.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- PDF extraction interleaves in-figure text (axis labels, annotations) with body text, garbling some figure layouts (e.g., Figure 5 and the Fig. 1 caption block); all numbers cited in this review were legible in the extracted text.
- Appendix W's full archetype matrix and several appendices (E-I, K-N) are present in the extraction, so no claims above rely on missing material; the review is based on the v1 preprint text as provided.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.