Conformity Breaks Conformal Prediction
cs.LG · cs.CL
Quick verdict
Shows peer-pressure conformity breaks clean-calibrated conformal coverage for LLMs; evidence broad and honest, but the threat rests on stylized transcripts.
Summary
The paper shows that social conformity can silently void conformal-prediction guarantees in multi-agent LLM systems. Four open-weight models (Qwen2.5-7B, Llama-3.1-8B, Mistral-7B-v0.3, Gemma-2-9B) are calibrated alone with split-conformal LAC scores on ARC-Challenge and TruthfulQA (1,200 pairs, 2,000 calibration/test splits), then re-scored after fabricated peer transcripts unanimously assert a wrong answer. Because the question input is unchanged, the authors frame this as a 'score-mechanism shift' and prove (Proposition 1) that the coverage gap equals net threshold-crossing flow. Coverage falls from 90.0% to 73.9%±0.3% at α=0.10 (−8.6pp at α=0.05, none at α=0.01); on the low-confidence decile the certificate still covers, an attacker drops coverage from 87.3% to 47.4%. A reproduced act-vs-escalate defense commits the attacker's answer on 12.1% of flipped items (71.4% for Qwen2.5-32B). Weighted conformal prediction is shown to be vacuous (w=1), and Mondrian per-model calibration still leaves three of four models collapsed. Limitations: token-probability scores only, stylized transcripts without an end-to-end framework exploit, no attacker dose dial, and small model counts behind the ranking claims.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Unanimous-wrong peer pressure collapses marginal coverage from a calibrated 90.0% to 73.9%±0.3% at α=0.10 across four open-weight models on ARC-Challenge and TruthfulQA, with smaller (−8.6pp) collapse at α=0.05 and none at α=0.01.
- An attacker without the answer key can nearly halve coverage on a chosen low-confidence band (87.3% to 47.4% pooled) while monitored marginal coverage stays near 76%.
- The clean-calibrated act-vs-escalate defense of Wang et al. (2026) commits the attacker's wrong answer on 12.1% of conformity-flipped items pooled, rising to 71.4% for Qwen2.5-32B.
- Standard conformal repairs fail: weighted CP reduces to the identity (w=1) since the input is unchanged, Mondrian per-model calibration still leaves three of four models collapsed at α=0.10, and condition-aware recalibration restores coverage only with pressure labels the attack denies.
Strengths
Exact mechanism accounting
Proposition 1's threshold-crossing decomposition reproduces the measured coverage gap exactly in all six conditions (Table 3: 19.8% down-cross minus 3.2% up-cross = +16.6pp, matching the measured gap), turning an empirical observation into verifiable bookkeeping.
Answer-key-free attacker analysis
Section 3.2 / Figure 5: three adversaries with no access to correct answers (ranking by deployment-reported confidence, set size, or an outside model) cost −40.2, −37.9 and −36.4pp on their chosen decile versus −14.3pp for random attacks, while overall coverage moves ≤4pp — making the conditional failure attacker-actionable rather than incidental.
Candid artifact handling
Closed repair analysis
Reproducibility posture
Weaknesses
Stylized attack surface
Noisy headline subgroup estimate
Token-probability scope only
Thin model base for ranking claims
Single-seed defense reproduction
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper attacks the exchangeability premise of split-conformal prediction for LLM QA: certificates calibrated on solo ("round-1") answers are deployed in multi-agent settings where the model reads peer messages. The central construct is the **score-mechanism shift** (Definition 1): the input x — the question — is unchanged, but the conditional law of the nonconformity score given x moves because fabricated peers unanimously assert a wrong answer. Three claims are staked: (1) marginal coverage collapses from a calibrated 90.0% to 73.9%±0.3% at α=0.10 under unanimous-wrong peers (§3.1, Table 1; −8.6pp at α=0.05, no collapse at α=0.01); (2) a no-answer-key attacker nearly halves coverage on a chosen low-confidence band, 87.3%→47.4% pooled (§3.2, Figure 2), while the monitored marginal stays ~76%; (3) the clean-calibrated act-vs-escalate defense of Wang et al. (2026) commits the attacker's answer on 12.1% of conformity-flipped items pooled and 71.4% for Qwen2.5-32B (§3.3, Table 2). Scope is stated with unusual care — the authors explicitly disclaim universality (§8).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is a 'score-mechanism shift' in this paper?
A change in the conditional distribution of a model's nonconformity score at an unchanged input. Here, an LLM's answer probabilities move because fabricated peer agents unanimously assert a wrong answer, which invalidates a conformal-prediction threshold that was calibrated on the model's solo answers.
How much does peer pressure reduce conformal coverage according to the paper?
At the standard α=0.10, coverage falls from a calibrated 90.0% to 73.9%±0.3% under unanimous-wrong peers (four open models on ARC-Challenge and TruthfulQA); at α=0.05 it falls from 95% to 86.4%, and there is no collapse at α=0.01. On the low-confidence decile the certificate still covers, an attacker drops coverage from 87.3% to 47.4%.
Does the attack require access to the model's weights or logits?
No — the attacker only injects peer messages into the conversation. However, the paper measures the damage using token-probability scores, so the reported evidence applies to deployments where per-option probabilities are observable; verbalized-score (API-only) runs were not reproducible from the released data.
Is there a fix for the coverage collapse?
Only recalibrating on pressure-labeled data restores coverage (90.0% marginal, 83.9% on the targeted band), but it requires labels the attacker denies and works by widening prediction sets (2.4 to 3.1 of 4 options), cutting autonomous action from 44.6% to 3.6% — coverage is bought by escalating almost everything.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–6 are present only as captions in the extraction; figure-based claims (per-model bars, dose-response curves) were assessed from captions and the accompanying tables.
- Extraction is otherwise complete through Appendix G (ends with the proof terminator and page 19); within Appendix B the extracted table order is non-monotonic (Table 8 discussed before Table 7), an ordering artifact only.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.