CompassOPD: Cross-Family On-Policy Distillation via Within-Family Likelihood Shifts
Naibin Gu, Qingyi Si, Chenxu Yang, Chuanyu Qin, Junhao Zhou, Peng Fu, Zheng Lin, Weiping Wang
cs.LG
Quick verdict
Claims cross-family OPD is masked by a family offset; CompassOPD removes it, gaining up to 5.50 pts. Well-ablated but math-only, no significance tests.
Summary
The paper studies why on-policy distillation (OPD) underperforms when teacher and student come from different model families, even after text-space tokenizer alignment. In motivating experiments (Figure 1), a Qwen3-4B student distilled from Qwen3.5-35B-A3B versus Qwen3.5-4B teachers differs by only 0.1 points despite a 16.7-point teacher gap, and small teacher-family models agree with a 35B teacher on about 70% of token-level update directions. The authors decompose the cross-family OPD signal into a cross-family offset between a low-capability teacher-family reference and the student, plus a within-family likelihood shift from reference to strong teacher, arguing the offset dominates updates. CompassOPD discards the offset, supervises with the teacher-family shift, and anchors updates to a frozen copy of the student's initial policy (advantage ΔT − αΔS, α=0.5), with a closed-form target policy derived in Appendix B. Across three student families (Granite4.1-3B, Qwen3-4B, OLMo-3-7B) and three teacher families, CompassOPD beats standard OPD on 20 of 21 student-benchmark pairs (Table 1) and 13 of 14 (Table 2), with average gains up to 5.50 points; an MoE self-reference built by reducing expert activation retains a 3.43-point gain. Limitations: math-reasoning-only evaluation, no confidence intervals or seed variance, no comparison to competing cross-tokenizer OPD methods, and no code release.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Cross-family OPD fails to track teacher capability because its signal is dominated by a static cross-family offset between a low-capability teacher-family reference and the student (Figure 1b: 16.7-pp teacher gap yields a 0.1-pp student gap).
- Removing the offset and supervising only the within-family likelihood shift ΔT, anchored to a frozen copy of the student's initial policy, beats standard cross-family OPD in 20/21 (Table 1) and 13/14 (Table 2) configurations, by up to 5.50 average points on Granite4.1-3B.
- Gains scale with teacher-reference separation (Table 3: 30.86 at 35B-A3B down to 26.50 at 2B, fixed 0.8B reference) and degrade monotonically as the offset is restored (Figure 3b), supporting the causal mechanism.
- An MoE teacher can serve as its own reference by reducing expert activation to ~2B active parameters, retaining a 3.43-point average gain over OPD without a separate checkpoint (Table 4).
Strengths
Clear diagnostic decomposition with motivating evidence
§3 grounds the method empirically: Figure 1(b) shows a 16.7-pp teacher gap (Qwen3.5-35B-A3B vs 4B) collapsing to a 0.1-pp student gap, and Figure 1(c) shows ~70% direction agreement between 0.8B and 35B-A3B scorers on identical trajectories — together motivating the O + ΔT split in Eq. 6 rather than a heuristic fix.
Causal ablations genuinely test the mechanism
Figure 3(b)/Eq. 12 restores the removed offset with coefficient β and accuracy declines monotonically on Granite4.1-3B; Table 3 shows gains shrink as teacher-reference separation narrows (30.86 at T=35B-A3B → 26.50 at T=2B, all with R=0.8B) yet 2B still beats OPD's 25.36.
Well-controlled comparisons
Practical MoE self-reference
Weaknesses
No uncertainty quantification
No comparison to existing cross-tokenizer OPD methods
Student-anchor confound not fully isolated
Math-only evaluation and untested reference sensitivity
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper targets cross-family on-policy distillation (OPD): distilling a teacher from a different release series (Qwen3.5-35B-A3B, Qwen3-30B-A3B-Instruct-2507, Ministral-3-14B) into students from other families (Granite4.1-3B, Qwen3-4B, OLMo-3-7B) after text-space tokenizer alignment. Central claims: (1) cross-family OPD plateaus because its token-level signal is dominated by a static "cross-family offset" — the likelihood gap between a low-capability teacher-family reference and the student — rather than by the strong teacher's capability (§3; Figure 1b: a 16.7-pp teacher gap yields only a 0.1-pp student gap; Figure 1c: ~70% direction agreement between 0.8B and 35B-A3B scorers on identical trajectories); (2) removing the offset and transferring only the within-family shift ΔT = ℓT − ℓR, with a frozen-student anchor −αΔS, recovers up to +5.50 average points (§5.2); (3) an MoE teacher can supply its own reference via reduced expert activation, retaining +3.43 (§5.4, Table 4). Claims are scoped to math reasoning.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is CompassOPD?
CompassOPD is a cross-family on-policy distillation method for LLMs. Instead of matching a cross-family teacher's absolute token likelihoods, it supervises the student with the log-likelihood shift between the strong teacher and a smaller reference model from the teacher's own family (removing the static cross-family offset), while a frozen copy of the student's initial policy anchors the updates.
Why does on-policy distillation fail across model families?
According to the paper, the token-level OPD signal is dominated by a fixed cross-family offset between a low-capability teacher-family reference and the student, so improvements in teacher capability barely change the update direction — the paper shows a 16.7-point teacher gap producing only a 0.1-point student gap, and ~70% direction agreement between small and large teachers within a family.
How much does CompassOPD improve over standard OPD?
On seven math benchmarks, CompassOPD improves average accuracy over standard cross-family OPD by up to 5.50 points (Granite4.1-3B student with the Qwen3.5-35B-A3B teacher and a 0.8B reference). When the reference is built from the MoE teacher itself by reducing expert activation, it still gains 3.43 points.
Does CompassOPD require a small reference model from the teacher's family?
Normally yes — a low-capability sibling of the teacher provides the within-family reference. For MoE teachers, the paper shows the reference can be constructed from the same checkpoint by activating fewer experts (~2B active parameters), which retains most of the benefit (28.79 average vs 30.86 with a dedicated 0.8B reference).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The abstract's final sentence is cut mid-phrase ('retaining a 3.43-point gain over') in the provided text; the completed claim appears verbatim in §5.4, so no substantive content is missing.
- Figure-embedded axis labels in Figures 1 and 3 extract as raw coordinate strings, so per-point values of the α-sweep (Fig 3a) and β-sweep (Fig 3b) curves cannot be read; the text states only the optimum (α=0.5) and the monotonic decline.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.