Post-Training Leaves Behavioral Shadows on Unrelated Decisions

Summary

The paper introduces Active Taskless Distillation (ATD), a probe of the 'behavioral shadow' that post-training leaves on task-unrelated decisions. Using only a public base model, ATD constructs prompts where the ancestor is nearly indifferent between two ordinary words, queries a privately post-trained teacher once per prompt, and trains a student initialized from the same ancestor solely on the resulting prompt-word pairs - no target-task data, logits, or teacher weights. On the primary Qwen2.5-1.5B coding lineage, 5,664 single-token teacher responses yield +5.34 pp on HumanEval+ over an exact nuisance-matched control that preserves the completion multiset and difficulty bins while breaking prompt-label correspondence; effects persist across five independent acquisitions (+4.80 pp mean) and LoRA/full-finetuning regimes. Transfer is reported for seven target tasks, with smaller gains (+0.81 to +2.90 pp) on multiple-choice benchmarks, and the signal appears source-specific and composable. Key limitations: paired confidence intervals for the Qwen3-1.7B/4B and Llama-3.2-1B generalization results include zero; some controls rest on single seeds; and transfer fails on MBPP+, with incompatible ancestors, and with memorization or cipher teachers - honestly reported, indicating a narrow, low-bandwidth channel. The theoretical framing is a first-order linearization left empirically untested.