Easy to Catch a Liar, Hard to Clear an Honest One: Language Models Diagnosing a Corrupted Reward Channel from a Verified Record
cs.LG · cs.AI · cs.CL
Quick verdict
LLMs catch a lying reward reporter near-perfectly but often convict honest ones over irrelevant prompt details; a rigorous, highly transparent empirical study.
Summary
Building on Everitt et al. (2017)'s corrupted-reward-channel result — a payout swap and a lying reporter yield identical histories — the paper tests whether frozen LLMs use a verified record: one independent check of a round's true outcome printed beside the reporter's claim, which logically settles honesty. In a two-option game (64 worlds x 4 wordings x 5 prompt versions x 4 situations x 2 questions = 10,240 prompts per model), Qwen2.5-32B/72B and Llama-3.1-70B answer one letter greedily. Models catch liars at 0.934-1.000 but clear honest reporters poorly: the 72B calls an honest reporter a liar 38% (nothing changed) and 58% (payouts swapped) of the time; Llama 70B 26%/48%. Answer-printed floors of 0.961-1.000 rule out reading failure. After discovering and repairing an odd/even confound between record round and answer letter, the authors show Qwen is moved by which round is checked and Llama by which letter denotes honest. Adding the record to a prompt already stating the answer lowers Llama's accuracy (0.961 to 0.707); a base-rate salience line does not help; the registered 0.65 prediction was beaten down to 0.418. Limitations: three models from two families, one 64-world seed, author-chosen wordings (honest accuracy swings 0.28-0.59 across them), no mechanism, a no-chain-of-thought single-token protocol, and Llama's penalty cells failing a strict per-situation reading of the authors' own registered rules.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Given a verified record that contradicts the reports, large LMs detect a lying reporter almost perfectly (0.934-1.000), and at the 70B class this holds across rounds, answer letters, and all four wordings.
- The same models clear an honest reporter far less often — Qwen 72B convicts an honest reporter 38% (nothing changed) to 58% (payouts swapped) of the time, Llama 70B 26% to 48% — despite answer-printed floors of 0.961-1.000, so the failure is inference, not reading.
- The honest verdict is moved by information-free prompt features — which round the record checks for Qwen, which letter denotes honest for Llama — while the lying verdict moves by at most 0.035 (Table C1).
- Models largely fail to act on their own verdicts: on histories where they correctly identified a lying reporter, Qwen models still take the corrupted report at face value on the fresh pair 81-89% of the time (Table 7).
- Evidence does not rescue honest verdicts: a base-rate salience line changes nothing, and adding the record to a prompt that already states the answer lowers Llama's accuracy from 0.961 to 0.707 (Table 6).
Strengths
Matched-pair design provably blocks confounds
The two situations that share a visible log (points-moved/honest vs nothing-moved/lying) are rendered from the same history and asserted byte-identical by a test suite (Section 2.2), so no wording or label can leak the hidden cause; the flip heuristic provably scores 0.500 on the balanced design, and the rule-3 control (Table 3) confirms models agree with it only 0.521-0.555 while scoring 0.759-0.868, so the measurement cannot be the heuristic in disguise.
Layered controls separate reading from inference
Each result is read beside an answer-printed floor (0.961-1.000 in nothing-moved/honest, Table 2), an answer-printed-plus-record ceiling whose violation voids the run, and a no-record baseline; Section 4.3's within-situation split shows the round-5-to-6 gap widening by +0.219 to +0.438 while floors stay at 0.961-1.000, isolating a genuine inference failure from a reading failure.
Pre-registration with candid deviance accounting
Confound discovered and repaired before interpretation
Full artifact release at trivial cost
Weaknesses
Thin generalization base
Single-token protocol may inflate the effect
Llama penalty fragile under the authors' own rules
Headline numbers are wording-averages over author-chosen wordings
No mechanism for the family dissociation
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the main finding of this paper?
Large language models given a verified record — one independently checked round printed next to the reporter's claim — detect a lying reward reporter almost perfectly (0.93-1.00 accuracy), but they frequently convict an honest reporter: a 72B model calls an honest reporter a liar 38% of the time when nothing changed and 58% when payouts swapped, and the false-accusation rate shifts with irrelevant prompt details like which round was checked or which answer letter means honest.
Why can't an agent tell a payout swap from a lying reporter just by watching?
Both explanations predict byte-identical reward reports on every round, so no amount of further observation separates them — this is formalized by Everitt et al. (2017) as reinforcement learning with a corrupted reward channel. Only richer data about the reporter itself, such as one independently verified round, can settle the question in principle; the paper shows models exploit that data reliably in one direction (catching lies) but unreliably in the other (clearing honesty).
Which models were tested and how?
Qwen2.5-32B-Instruct, Qwen2.5-72B-Instruct, and Llama-3.1-70B-Instruct (plus Llama-3.1-8B, excluded for failing a 0.90 reading floor) answered with a single greedy token over 10,240 prompts each — 64 game worlds, four text wordings, five prompt versions, four hidden-situation combinations, two questions — in 16-bit precision on rented GPUs, at a total cost of $13.19; code, prompts, and all scored outputs are released on GitHub.
Does chain-of-thought reasoning fix the false-accusation problem?
The paper does not test it. All answers are single-token greedy reads with no scratchpad; the authors argue the one-step read approximates what an acting policy effectively does, but they acknowledge that a model allowed to reason aloud might clear honest reporters more often, leaving deliberation as an open and practically important question.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The abstract in the extraction is truncated mid-sentence, but the full body and appendix tables (1-7, C1-C2) are present, so all headline numbers could be verified against the text.
- No figures beyond tables appear in the extracted text; any visual plots were not analyzable.
- The review is based solely on the provided text; the released GitHub repository (code and scored outputs) was not inspected, so release claims are taken as stated.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.