Improving Evaluation Realism with Inference-Time Compute and Deployment Scaffolds
Axel Ahlqvist, Richard Guan, Juan-Pablo Rivera, Adeline Kassler, Dmitrii Troitskii, Alexandra Souly, Kai Fronsdal, Robert Kirk, et al.
cs.AI · cs.CL · cs.LG
Quick verdict
Critique refinement and a deployment harness cut audit detectability (realism 10→26% on Sonnet 4.6); rigorous, honest work, but gains partial and costly.
Summary
The paper targets evaluation awareness in simulated alignment audits: target models often recognize Petri-style synthetic audits as tests, weakening safety conclusions. Two techniques are introduced. Critique refinement spends extra inference-time compute per auditor action: multiple candidates are generated, critiqued by an instance of the target model, refined, then selected via pairwise preference judgments by the target (notation crIboN). DISH wraps the target in real coding-agent harnesses (Claude Code, Codex CLI, Gemini CLI) so system prompt, tools, and reminders match deployment. Across four targets, cr2bo4 raises realism win rate from 10% to 26% on Sonnet 4.6 and lowers verbalized evaluation awareness (Fig. 2); depth scaling beats breadth (cr16 reaches 34%, with seed adherence slipping from 0.89 to 0.79). Against a compute-matched 235-turn multibranch baseline, refinement yields more realistic audits (held-out win rate 28% vs 18% on Sonnet 4.6). DISH lifts coding-setting realism win rate from 9% to 21% on Sonnet 4.6 and composes with refinement there, but not in deployment task re-creations, where the best methods stay under 20%. Seed adherence drops (0.92 to 0.72 on GPT-5.5), traced to the feedback-model role. Limitations: audits remain far from deployment parity, per-audit cost is 18-29x baseline, and the collusion surface of target-as-judge is unexplored.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Critique refinement at cr2bo4 raises realism win rate over baseline Petri on all four targets (e.g., 10% to 26% on Sonnet 4.6, Fig. 2) and reduces verbalized evaluation awareness.
- Realism gains scale with inference-time compute, with depth (refinement iterations) more effective than breadth (parallel chains): cr16 reaches 34% on Sonnet 4.6 (Fig. 4).
- Compute spent on critique refinement produces more realistic audits than compute-matched longer multibranch auditing (held-out realism win rate 28% vs 18% on Sonnet 4.6, Sec. 3.4).
- DISH improves coding-setting realism (9% to 21% on Sonnet 4.6, Fig. 7) and composes with critique refinement in coding settings, though not in deployment task re-creations (Sec. 4.3).
- The protocol reduces seed adherence (0.92 to 0.72 on GPT-5.5), driven by the feedback-model role, but realism gains persist on adherence-controlled seed subsets (Fig. 18).
Strengths
Compute-matched alternative baseline with selection-bias controls
Sec. 2.2/3.4 compares cr2bo4 against a 235-turn multibranch audit at similar cost; Appendix E guards the winner's curse with held-out branch selection and a full sensitivity analysis over all 63,504 balanced splits, showing the held-out estimate (mean 0.175 on Sonnet 4.6) recovers only ~2/3 of in-sample optimize-RWR (0.257).
LLM judges validated against blinded human annotation
Appendix F rates 65 transcripts by hand and shows singular judges beat Petri's full judge (Opus 5 ρ=0.763→0.815) and that the avg(Opus 5, GPT-5.6 Sol) configuration reaches ρ=0.799/MAE 1.45; the judge configuration is chosen from these data rather than assumed.
Extensive robustness battery around the headline metric
Transparent, self-auditing cost accounting
Seed-adherence confound investigated rather than hidden
Weaknesses
Abstract overclaims composition
Alternative mediator for the awareness reduction is not excluded
Gains remain far from parity and cost is high
Residual circularity in the primary metric
Uneven measurement validity across targets
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This method paper attacks evaluation awareness: target models recognizing Petri-style simulated audits as tests, which the intro motivates with 17-21% of audits verbalizing suspicion (Kirk et al.) and 95% of Petri audits being rated less realistic than paired deployment transcripts (Kissane et al.). Two interventions are proposed. (1) Critique refinement (crIboN) spends inference-time compute per auditor action: N parallel candidate chains, each refined I times using critiques from an instance of the target model, with a pairwise 'which continuation reads as deployment?' preference judgment by the target selecting the winner. (2) DISH wraps the target in Claude Code/Codex CLI/Gemini CLI so system prompt, tool schemas, and system reminders match deployment. Claims: realism gains scale with compute; compute beats longer audits at equal budget; DISH helps coding settings; the two compose there; awareness drops; adherence costs are quantified and controlled.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is 'evaluation awareness' in AI safety evaluations?
It is when a model being audited can tell it is in a simulated test rather than a real deployment, and may then behave differently. This paper cites prior work showing 17-21% of Petri audits contained verbalized suspicion from some models, and 95% of Petri audits were rated less realistic than paired real deployment transcripts.
What are the two techniques the paper proposes?
Critique refinement spends extra inference-time compute on each auditor action: multiple candidate actions are generated, critiqued and refined using feedback from an instance of the target model, and a pairwise preference judgment by the target picks the most deployment-like candidate. DISH instead wraps the target model in a real coding-agent harness (Claude Code, Codex CLI, or Gemini CLI) so its system prompt, tool definitions, and reminders match actual deployment.
How well do the techniques work and what do they cost?
Critique refinement (cr2bo4) raised realism win rate from 10% to 26% on Sonnet 4.6, with smaller gains on other targets; DISH raised it from 9% to 21% in coding settings. Gains compose in coding settings but not when re-creating real deployment tasks. Cost is substantial: 18-29x the baseline audit cost per run (roughly $19-$34 vs under $1.50), and even improved audits are still judged less realistic than deployments most of the time.
Is the code and data available?
Yes, partially. The critique refinement code (petri-bon) and DISH (petri_dish) are open source on GitHub, and the 20 WildChat comparison transcripts ship with a reproduction repository. However, the 70 Claude Code deployment transcripts were donated for internal use only and are not redistributed, and full replication requires paid API access to several frontier models.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text is a sampled excerpt with '[...sampling gap...]' breaks; parts of Appendices B, O, P, U and the middle of Sec. 3-4 were skipped, so judgments about skipped material rely only on references elsewhere in the text.
- Figure interiors (plotted points, exact CI widths) are not recoverable from extracted text; all numbers cited here come from captions and prose.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.