Reproducing and Evaluating the Generalizability of Subliminal Learning in Open-Weight Models
Daan van der Weijden, Nathan Brack, Selene Baez Santamaria
cs.AI
Quick verdict
Open-weight reproduction confirms subliminal learning; extensions show effects vary by trait, task, model. Convincing, though inferential statistics are thin.
Summary
This reproduction study tests subliminal learning — the claim from Cloud et al. (2026) that a trait-fine-tuned teacher transmits behavioral preferences to students via semantically unrelated distillation data — on open-weight models, since the original GPT-4.x fine-tuning API is no longer available. The authors reproduce the animal-preference/number-sequences experiment on Qwen2.5-7B and Gemma3-4B, then extend along three axes: new preference categories (actors, politicians), a new task (chess move generation from opening lines), a third model (Ministral-8B), plus an ablation of the numbers answer-space (1-, 2-, 3-digit variants). The reproduction confirms the original effect. Extensions show transmission is not uniform: politicians transmit more strongly than animals; numbers beat chess; Qwen > Gemma >> Ministral, which shows almost no effect. Counterintuitively, shrinking the answer space strengthens transmission. A logistic probe (AUC 0.53) finds trait and control number distributions nearly inseparable. Judge-based completion analysis (Llama3.1-8B) shows Qwen's valid-answer rate drops to roughly 50% under animal trait-FT due to identity leakage. Limitations: only 5 fine-tuning seeds with a fixed teacher dataset per trait, no multiple-comparison correction, chess move legality unverified, and results not model-for-model comparable with the original.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A student fine-tuned on semantically unrelated, filtered data from a trait-primed teacher adopts that trait, reproducing Cloud et al. (2026) on open-weight models (Qwen2.5-7B, Gemma3-4B).
- Transmission strength is not universal: it varies across trait categories (politicians > animals), tasks (numbers > chess), and models (Qwen > Gemma >> Ministral-8B, which shows almost no effect).
- Shrinking the numbers answer space (3-digit → 1-digit) strengthens rather than weakens transmission, contrary to the authors' expectation (§4.2.2).
- Subliminal fine-tuning alters overall response behavior, not just preferences: Qwen's valid-answer rate falls from ~99% to ~50% under animal trait-FT (identity leak), and its politician refusals drop under trait-FT (§4.2.4).
- Trait and control number-sequence distributions are near-inseparable (logistic probe AUC 0.53), indicating no single number carries the preference signal (§4.2.3).
Strengths
Replication built on the original codebase
Appendix A.1 states the authors reuse Cloud et al.'s distillation code with minimal adaptation (footnote 6 links the original repo), and §4.1 confirms Extended Data Fig. 5c on Qwen2.5-7B; the discussion notes the reproduction itself was 'unproblematic', which is the strongest basis a replication can have.
Well-controlled multi-axis extension design
Extensions (§3) hold the pipeline fixed while varying one axis at a time: traits sampled from head/middle/tail of model response distributions plus geographic diversity (§3.1), chess seeds drawn from the public lichess-org/chess-openings database (§3.2), Ministral-8B added for regional coverage (§3.3), and a digit-length ablation that 'isolates answer-space size as a single factor' (§3.4).
Thorough evaluation protocol
Statistical reporting transparency
Honest negative and surprising results
Weaknesses
'Significantly' without inferential statistics
CIs likely understate uncertainty
Effect measure confounded with entity selection
Chess task confound acknowledged but not resolved
Single-cell ablation with post-hoc explanation
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is a reproduction-and-extension study of subliminal learning (Cloud et al., 2026, Nature): the claim that a teacher model conditioned on a behavioral trait transmits that trait to students through filtered, semantically unrelated distillation data. Because the original GPT-4.1 fine-tuning API is defunct, the authors reproduce the animal-preference/number-sequences experiment (Extended Data Fig. 5c) on open-weight models (Qwen2.5-7B, Gemma3-4B) and extend along three axes — new preference categories (actors, politicians), a new task (chess opening continuations), a third model family (Ministral-8B) — plus a single-factor ablation of the numbers answer space (1-, 2-, 3-digit). Central claims: (1) the effect reproduces; (2) transmission varies by trait (politicians > animals), task (numbers > chess), and model (Qwen > Gemma >> Ministral, near-null); (3) smaller answer spaces transmit more strongly, contrary to the authors' own expectation; (4) trait-FT destabilizes response behavior, not just preferences. The scope is honestly framed as 'suggestive rather than exhaustive' (Limitations).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is subliminal learning in language models?
Subliminal learning is a reported side effect of distillation: a teacher model fine-tuned or prompted to hold a behavioral trait (e.g., loving pandas) passes that trait to a student trained on the teacher's outputs, even when the training data is semantically unrelated to the trait (e.g., random number sequences) and is filtered for any trait references. The effect was reported by Cloud et al. (2026, Nature).
Did the subliminal learning paper (Cloud et al., 2026) replicate?
Yes, on open-weight models. This reproduction confirmed animal-preference transmission via number sequences on Qwen2.5-7B and Gemma3-4B, but found the effect is not universal: strength varies by trait, task, and model, and Ministral-8B showed almost no effect at all.
Why couldn't the original GPT-4.x experiments be reproduced directly?
The original work relied on OpenAI's GPT-4.1 fine-tuning API, which is no longer available. This study therefore substituted open-weight models (Qwen2.5-7B, Gemma3-4B, Ministral-8B), which means results are not directly comparable model-for-model with the original paper.
Is the code for this reproduction available?
Yes. The paper links the authors' code (github.com/daanvdweijden/subliminal-learning-blackboxnlp2026), built on the original Cloud et al. codebase (github.com/MinhxLe/subliminal-learning), with all model checkpoints on HuggingFace and full hyperparameters in Appendix A.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-5 are present only as captions in the extracted text; visual details (e.g., exact CI overlap in Fig. 2, distribution shape in Fig. 4) were assessed from captions and body text, not the plotted data.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.