On Repulsive and Attractive Teachers: Separating Correctness from Behavior in Self-Distillation
Anton Baumann, Akmal Ashirmatov, Leo Schmidt-Traub, Frederike Lübeck, Jonas Hübotter, Thomas Kleine Buening, Andreas Krause
cs.LG · cs.AI
Quick verdict
Shows repulsive self-distillation mainly shifts behavior; contrastive SD is stable alone without GRPO, but evidence is single-model (4B), figure-based.
Summary
On-policy self-distillation conditions a model on privileged information (such as a correct solution) to create a teacher, which then rescores the student's own rollouts token by token. The authors show this conditioning changes not only what the teacher knows but how it behaves. In experiments with Qwen3-4B variants on DeepMath and Reasoning Gym's Group Anagrams, attraction toward a correct-solution teacher suppresses exploration and shortens responses, while sign-reversed repulsion (KL ascent) drives uncontrolled response-length growth: hybrid models switch into their latent thinking mode without beating the thinking-mode baseline, and models without a thinking mode eventually collapse under truncation. Motivated by this, they study contrastive self-distillation—attraction toward a correct-solution teacher plus repulsion from an incorrect-solution teacher—without GRPO. At λ=1/2 the student term cancels exactly and the teachers' shared behavioral shifts largely cancel, leaving a correctness-dominated token-level signal that improves all three settings (non-thinking, instruct-only, already-thinking) while keeping length stable. Ablations show token-level credit assignment is crucial (permuting advantages drops Group Anagrams training accuracy from 85.5% to 62.2% at step 50) and privileged context governs thinking length. Limitations: single 4B model family, two task domains, no multi-seed variance or confidence intervals, and headline gains presented only in figures.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Conditioning a self-teacher on solutions shifts student behavior largely independent of correctness: attraction yields short, confident, less exploratory responses; repulsion yields longer, more uncertain ones.
- One-sided repulsion without GRPO primarily induces behavioral change — re-activating a latent thinking mode in hybrid models or inflating response length — and ends in truncation collapse rather than teaching new capability.
- Contrastive self-distillation at λ=1/2 cancels the student term exactly (Eq. 8) and largely cancels the teachers' shared behavioral shifts, leaving a correctness-dominated token-level signal.
- The GRPO-free contrastive objective improves reasoning across non-thinking, instruct-only, and already-thinking models while keeping response lengths stable (Sections 5.1-5.3).
- Fine-grained token-level credit assignment is essential: permuting advantages within responses cuts Group Anagrams training accuracy from 85.5% to 62.2% at step 50 (Section A.2).
Strengths
Clean isolation of a confounded signal
First controlled study of repulsive and contrastive self-distillation without the GRPO term that AntiSD, Rebellious Student, CEPO, and RLCSD all rely on (Sections 1-3), making attribution to the distillation signal itself possible.
Falsification chain for the mode-switch hypothesis
Sections 5.1-5.3 plus A.1 triangulate: repulsion lifts a hybrid non-thinking model into its dormant thinking mode without beating the thinking-mode baseline (Fig. 4), gives no gain on an instruct-only model (63.8%→63.6%→64.2%, then 100% truncation by step 29, Fig. 7), and still collapses when thinking is active from step 0 (Fig. 6).
Exact analytical cancellation with empirical support
Mechanism-probing ablations beyond headline accuracy
Above-average reproducibility disclosure
Weaknesses
No variance or significance reporting
Headline results are figure-only
Narrow external validity
λ weighting unexamined
Stability horizon and data-distribution caveats
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This ETH Zurich preprint (arXiv:2609.21561) asks a narrowly scoped but consequential question for the post-training community: what does the sign-flipped ("repulsive") self-distillation signal actually do when stripped of the GRPO reward that always accompanies it in AntiSD and Rebellious Student, and can a contrastive pairing of correct- and incorrect-solution teachers stand alone? The setting is deliberately compact: Qwen3-4B (non-thinking and thinking modes) and Qwen3-4B-Instruct-2507, trained on a 10,000-problem hard subset of DeepMath (difficulty ≥ 8) and on Group Anagrams from Reasoning Gym (2,200 train / 400 eval; expert solutions generated with Qwen3-235B-A22B-FP8, Section B.1). Three objectives from one family (Equation 6): attractive λ=1, repulsive λ=0, contrastive λ=1/2. The headline claims are the three takeaways: repulsion mainly shifts behavior rather than teaching capability; apparent recovery in hybrid models is a dormant-mode switch; and the contrastive objective is stable and improves all three model configurations without GRPO.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is repulsive self-distillation and why does it fail?
Repulsive self-distillation reverses the sign of the distillation objective: instead of moving the student toward a teacher conditioned on privileged information such as a correct solution, it moves the student away (KL ascent). Methods like AntiSD and Rebellious Student use it to encourage exploration, but always together with a GRPO reward. This paper shows that on its own, repulsion mainly changes behavior — lengthening responses, re-activating a dormant thinking mode in hybrid models — and becomes unstable, collapsing once responses hit the token cap.
What is contrastive self-distillation and why does it work better?
It combines two self-teachers built from the same model: attraction toward one conditioned on a correct solution and repulsion from one conditioned on an incorrect solution, weighted equally (λ=1/2). The student term cancels exactly, and because both teachers share the same behavioral biases (concise, confident style), those cancel too, leaving a token-level signal that mostly reflects correctness. Trained without GRPO, it improves math and puzzle reasoning across non-thinking, instruct-only, and thinking-mode models while keeping response length stable.
Which models and datasets were used in this paper?
Qwen3-4B run in non-thinking and thinking modes, plus Qwen3-4B-Instruct-2507 (no thinking mode). Training used a 10,000-problem hard subset of DeepMath-103K (difficulty ≥ 8) and the Group Anagrams task from Reasoning Gym (2,200 training problems). Evaluation covered AIME 2024/2025/2026, AMC23, HMMT25, and a 128-problem MATH-500 subset, plus four Group Anagrams difficulty ranges, with four samples per problem.
How is this paper different from AntiSD, CEPO, or RLCSD?
Those methods combine repulsive or contrastive distillation signals with a GRPO reinforcement objective, so it is unclear what each component contributes. This paper isolates the distillation signal itself, showing that one-sided repulsion without GRPO is behaviorally distorting and unstable, while the balanced contrastive objective is stable and effective on its own — no verified-reward term needed.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-9 carry the main quantitative evidence; only captions and in-text descriptions were extractable, so curve shapes and endpoint accuracies could not be independently verified.
- The extracted text appears complete (Sections 1-6, Appendices A-B, Tables 1-4), so few content gaps are expected.
- Run/seed counts are never stated in the text; the absence of multi-seed reporting is inferred from total silence on variance.
- An accompanying blog post is referenced but was not part of the reviewed text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.