What Should a Self-Teacher See? Privileged Context Design for On-Policy Self-Distillation
Kanghui Tian, Siyuan Liu, Tianxiang Jiang, Shuai Dong, Yizhuo Li, Tian Ding, Yuan Guo, Songze Li, et al.
cs.LG · cs.AI
Quick verdict
声称较少的特权教师上下文在 OPSD 数学任务中于 4B/8B 峰值下优于完整解决方案；层级不稳定，需要种子复制。
Summary
本文询问在策略自蒸馏（OPSD）中，自教师应看到何种特权上下文，其中基础模型的冻结副本在给定额外上下文的情况下重新评估学生的 rollout。从完整参考解决方案（L1）出发，离线编译器构建了一个命名策略（L2）、一个与方法无关的框架（L3）和一个问题类别（L4），并设有仅答案对照（L5）。在三种种子的三个 Qwen3 规模（1.7B/4B/8B）中，每个规模均在 29,434 道竞赛数学题上进行了训练。在主要运行中，最佳中间上下文分别将 4B 和 8B 的域内峰值平均值提高了 +1.39 和 +1.57 分，同时存储的提示 token 减少了一个数量级；种子复制支持在四种检查点规则下，框架和类别条件在这两种规模下均带来正向平均增益。在 1.7B 时，完整解决方案仍然获胜，且没有任何上下文能赢得所有迁移基准。仅答案条件保持在 4B/8B 完整解决方案的 0.2 分以内，但在 1.7B 时落后 1.76 分。初始教师-学生 KL 散度无法预测下游性能。局限性：许多条件依赖于单次运行、相对于检查点变异性适度的峰值增益，以及限制在 Qwen3 和一个数学池上的实验。
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 4
● low 1
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- In OPSD on competition mathematics, intermediate privileged contexts (L2-L4) beat the complete reference solution at 4B and 8B in-domain peak means by +1.39 and +1.57 points (primary runs) while storing 17-37x fewer hint tokens.
- Across three training seeds, L3 (framing) and L4 (category) show positive mean paired differences over L1 at 4B and 8B under all four checkpoint aggregation rules, whereas at 1.7B the full solution retains the lead.
- Answer-only teacher conditioning stays within 0.2 points of the full solution's in-domain peak mean at 4B and 8B but trails by 1.76 points at 1.7B.
- The full solution induces the largest initial teacher-student KL at every scale, and this distributional displacement does not order downstream teaching utility.
- The preferred context level varies with student scale and task, and no context wins the whole transfer suite (Table 2).
Strengths
Unusually disciplined causal framing of an ablation axis
Section 3 treats the teacher context package as the intervention (Eq. 1-4), holds the student view, frozen teacher, data order, rollout procedure, and loss fixed within each scale, and explicitly separates estimands for peak vs. selection-free rules (Eq. 2, Sec. 3.4) — including an honest admission that the ID peak mean 'need not represent any single checkpoint'.
Multi-seed replication of the core contrast
L1/L3/L4 are retrained with seeds 42/43/44 at all three scales and aggregated under four checkpoint rules (Table 3): at 4B and 8B both L3 and L4 show positive mean paired differences under every rule (e.g., 4B all-checkpoint mean: L3 +1.03±0.15, L4 +1.01±0.32), and L4 is positive in all three seeds at both scales.
Audits instead of assumed contracts
Negative results reported prominently
Layered controls against alternative explanations
Weaknesses
Headline gains rest on per-benchmark peak selection
Ordering within L2-L4 is unstable
Many load-bearing cells are single runs
Semantic level is confounded with length and answer access
External validity confined to one family and domain
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks a design question inside on-policy self-distillation (OPSD, Zhao et al. 2026a): when a frozen copy of the base model rescores the student's own rollouts under privileged context, how detailed should that context be? Five levels are compared: L1, the complete reference solution (mean 657.3 Qwen3 tokens); L2-L4, offline-compiled abstractions (a named strategy ~38.9 tokens, a method-independent framing ~25.2, a problem category ~17.7) generated coarse-to-fine under contracts that prohibit the answer; L5, the bare final answer. The evaluation trains Qwen3-1.7B/4B/8B with LoRA for 200 steps on 29,434 competition math problems and reads out Avg@12 on AIME24/25 and HMMT25 (peak and selection-free rules), four transfer benchmarks, and a C-Eval audit.
The claims are calibrated to exactly this pipeline. Headline numbers come from the primary single-seed runs: best intermediate beats L1 by +1.39 (4B, L3 64.81 vs 63.43) and +1.57 (8B, L2 68.24 vs 66.67) ID peak points. Three-seed replication (Table 3) supports the family-level contrast: L3 and L4 both show positive mean paired differences over L1 at 4B and 8B under all four checkpoint rules, with the tightest evidence under the selection-free all-checkpoint mean (4B: +1.03±0.15 and +1.01±0.32). At 1.7B the effect reverses or vanishes. The KL result — L1 has the largest teacher-student displacement at every scale without ordering utility — cleanly refutes the paper's own 'discrepancy-based ordering' prediction.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does the paper claim about on-policy self-distillation and teacher context?
In OPSD, a frozen copy of the base model scores the student's own rollouts while seeing privileged context (normally the complete reference solution). The paper shows that coarser contexts — a named strategy, a method-independent framing, or a problem category — can train better students than the full solution at 4B and 8B (+1.39/+1.57 in-domain peak points in primary runs), while at 1.7B the full solution remains best.
Does giving the self-teacher only the final answer work?
Surprisingly well at larger scales: answer-only conditioning (L5) lands within 0.2 points of the full solution's in-domain peak mean at 4B and 8B, though it trails by 1.76 points at 1.7B. The best intermediate contexts still exceed L5 at the larger scales.
Which context level is best overall?
There is no universal winner. The full solution wins at 1.7B; L3 leads the primary 4B runs and L2 the primary 8B runs; and across seeds the L3-vs-L4 leader flips within both 4B and 8B. The stable finding is only that intermediate levels beat the full solution at 4B/8B on average.
Is the code or data for this paper available?
The paper states that compiler prompts, audit artifacts (sample IDs, raw outputs, adjudications), and code will be released with publication, but no repository link appears in the preprint text reviewed here; the training pool is the public 29,434-problem OPSD set from Zhao et al. (2026a).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-3 and 6-11 were only available as embedded text labels; judgments about them rely on the numbers stated in captions and body text, not visual inspection.
- The provided text appears essentially complete (main text plus Appendices A-E), but minor truncation of figures cannot be ruled out.
- The context audits are model-assisted (Qwen3.5-397B as both annotator and adjudicator); no human inter-rater agreement exists, so leakage rates carry model-judge bias.
- Claims about single-run cells (L2, L5, transfer, controls) are single observations; review treats them as such per the authors' own framing in Sec. 3.4.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.