ObserverBench: Testing Mechanistic Estimates for Intervention and Control

Summary

ObserverBench is a benchmark framework for testing whether internal mechanistic estimates—'observers'—are adequate for the interventions, control loops, or safety policies they guide. Each task pins the model, information boundary, allowed actions, decision rule, held-out cases, and loss, and reports estimation accuracy separately from downstream action loss. A finite-horizon certificate (Eq. 1) decomposes control error into reported error, starting-state mismatch, and response mismatch; on a two-block Transformer it correlates 0.952 with final error. On GPT-2-small IOI, pairwise observers improve mean-effect MAE (0.486→0.395) yet yield no reliable action gain, while direct-loss-trained observers cut action loss 18.5% versus mean plug-ins and 10.9% versus exact no-op at a pilot-informed target. The pattern transfers to a Qwen2.5-7B induction-copy task (MAE 0.121→0.040; 8.9% action-loss reduction). Safety experiments show a monitor with AUROC 1.000 allocates a fixed budget poorly when consequence is ignored, and that Qwen's own output scores nearly match an oracle. Matched APPS replications on Gemma-2-9B-it and Qwen3.5-9B show monitor rankings reverse across checkpoints, and SAE probes trail dense controls under disclosed density/checkpoint mismatches. Limitations: locally sealed rather than independently preregistered protocols, narrow pinned models and prompt distributions, no adaptive attacker.