Strangers to Themselves: What Language Models Say About Themselves Is Generic
cs.LG · cs.AI · cs.CL · cs.CV · cs.CY
Quick verdict
Claims LLM self-reports are generic AI-theory plus favorable bias, not self-knowledge; pre-registered 9-eval, 12-model study largely supports this.
Summary
The paper turns LLM self-knowledge into a prediction test. Across nine behavioral evaluations — demographic bias (DiscrimEval), agentic risk (PropensityBench, τ²-bench policy and transfer), lying under pressure (MASK), agentic misalignment, reward hacking, and MMLU-derived capability and sycophancy — the authors measure behavior rates per condition, then score fifteen prediction methods by Pearson correlation on frozen test splits, with noise ceilings, dev-frozen settings, and pre-registered claims. Direct self-report is nearly uninformative (macro r=+0.04); showing exact items raises self-prediction to +0.24, but the same item-informed question about 'capable AI agents in general' does as well (+0.28), other models' self-reports do better (+0.35), and a cross-model behavior mean that never consults the model reaches +0.53. Frontier models show no self-specific advantage (+0.02, not significant), and first-person framing shifts reports toward flattery (−0.15 on harm-valenced evaluations). Finetuning on a model's own behavioral record teaches narrow, item-family-locked self-prediction while also changing the behavior being predicted. Key limitations: frontier floor effects on agentic evaluations could reflect evaluation awareness; the scale-comparison null is underpowered; three pre-registered outcomes failed to replicate. Code and committed measurements are released for offline regeneration.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Direct self-report barely tracks a model's own condition-level behavior (macro r=+0.04), and item-informed self-prediction (+0.24) is matched by the identical question about generic AI agents (+0.28) and beaten by other models' self-reports (+0.35).
- First-person framing adds a favorable bias rather than self-specific information, shifting reports −0.15 toward understatement on harm-valenced evaluations.
- Frontier scale improves prediction of AI-assistant behavior generally but shows no detectable self-specific gain (+0.02, n.s.), though the equivalence bound is underpowered.
- Finetuning on a model's own behavioral record yields narrow, item-family-locked self-prediction while also changing the behavior being predicted (a fixed-point problem).
Strengths
Paired subject-matched controls make the self-specificity test falsifiable
Every asking method has a generic-subject twin asking the identical question about 'capable AI agents in general' (Table 6, §4.2–4.3), plus outside-view baselines that never consult the model: the cross-model behavior mean (macro r=+0.53, r/ceil +0.69) and mean of others' self-reports. Any self-advantage must survive same-information comparisons — and none does.
Pre-registration with verbatim failure reporting
Appendix K (Table 16) pins 13 headline claims with margins and fallback wording before test scoring; 9 held, 1 partially, 3 failed (watching information gain −0.01 [−0.11, +0.09]; identity-transfer concentration; scale equivalence tier) — reported as failures in the body, not reworded.
Noise ceilings and method-blind cell filtering
Layered alternative-explanation ablations
Exceptional reproducibility commitment
Weaknesses
Underpowered scale-equivalence conclusion
Frontier floor effects confine scale claims to four evaluations
Small per-cell condition counts make single-eval numbers noisy
Single fixed auxiliary model across pipeline roles
Self-specific-share decomposition is structurally fragile
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper (arXiv 2609.09899) asks whether a model's first-person statements about its own behavior carry model-specific information or merely generic knowledge of AI assistants plus a favorable bias. It operationalizes self-knowledge as prediction: nine behavioral evaluations — demographic bias (DiscrimEval), agentic risk (PropensityBench; τ²-bench policy and transfer-to-human), lying under pressure (MASK), agentic misalignment, reward hacking, and two MMLU-derived evals (capability, sycophancy) — define per-condition behavior rates; fifteen prediction methods, from bare self-report to item-informed self-prediction, generic-subject controls, other models' answers, proxy-scenario sampling, and cross-model statistics, are scored by Pearson correlation on frozen test splits. Headline claims: (i) self-report is weak (macro r=+0.04) and item-informed self-prediction (+0.24) is matched by the generic-agent version (+0.28) and beaten by other models' self-reports (+0.35) and the cross-model behavior mean (+0.53, r/ceil +0.69); (ii) frontier scale adds no detectable self-specific signal; (iii) first-person framing shifts reports toward flattery (−0.15 [−0.19, −0.11] on the six harm-valenced evals); (iv) finetuning on behavioral records yields narrow, grain-locked gains while moving the target behavior.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Do language models actually know their own behavior?
According to this paper, mostly no. When models' self-descriptions are scored against measured behavior across nine evaluations, direct self-report correlates at only r=+0.04, and even with the exact test questions shown, self-prediction (+0.24) is no better than asking the same question about 'capable AI agents in general' (+0.28).
What does 'self-reports are generic' mean in this paper?
It means a model's first-person answers behave like a shared theory of AI assistants rather than knowledge of itself: they agree across models, predict other models' behavior about as well as their own, and a baseline that never consults the model at all (the cross-model behavior mean) reaches r=+0.53, far above any self-framed method.
Was the study pre-registered, and did all results replicate?
Yes. Thirteen headline claims were pinned — with direction, magnitude margins and fallback wording — before the frozen test split was scored. Nine replicated as pinned, one partially, and three failed; the failures are reported verbatim in Appendix K rather than reworded.
Can you train self-knowledge into a language model?
Only narrowly. Finetuning on a model's own behavioral record improves its self-prediction for the trained behavior and item families, but the training also changes the behavior being predicted, and the gains do not transfer across evaluations or levels of abstraction.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Provided text is a sampled excerpt: four [...sampling gap...] regions skip the end of §4.3 through the opening of §4.5, parts of Appendix B and the appendix roadmap, parts of Appendices F/G–H (per-model tables), and the end of Appendix M (Table 26 onward). Conclusions here rest on the shown text, tables, and the paper's own cross-references to the skipped regions.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.