What Should a Self-Teacher See? Privileged Context Design for On-Policy Self-Distillation

Abstraction level, not information volume, governs the effectiveness of privileged context in on-policy self-distillation

Kanghui Tian; Siyuan Liu; Tianxiang Jiang; Shuai Dong; Yizhuo Li; Tian Ding; Yuan Guo; Songze Li; Haowen Hou; Congcong Wang; Yi Wang· 2026· DOI 10.48550/arXiv.2609.25623

The core problem

On-policy self-distillation (OPSD) leverages a frozen copy of the base model as a teacher that scores the student's own rollouts under a privileged context. Conventionally, this privileged context is a complete reference solution that bundles the final answer with one particular reasoning path. The central tension explored in this work is that more privileged information does not always yield a better teacher. The authors hypothesize that the optimal level of abstraction in the privileged context depends on what the student can still act on. They systematically compare the default full-solution context against three offline-compiled abstractions—a named strategy, a method-independent framing, and a problem category—as well as an answer-only control that preserves the destination but removes the reasoning path. The study aims to identify how context design interacts with student scale and task type, and whether initial teacher-student KL divergence can predict downstream performance.

Innovation

In the primary runs on competition mathematics, the best intermediate contexts improve the in-domain peak mean over the full solution by 1.4 points at 4B and 1.6 points at 8B. These gains are achieved while storing an order of magnitude fewer hint tokens. Across three seeds, the framing and category contexts show positive mean gains at both scales. Answer-only conditioning remains competitive, staying within 0.2 points of the full solution at these scales. The preferred context varies with student scale and task: for example, at 4B the named strategy context performs best, while at 8B the method-independent framing yields the highest peak mean. Initial teacher-student KL divergence does not order downstream performance, indicating that it is not a reliable predictor of context effectiveness. The results suggest that the optimal abstraction level is student-dependent and that providing too much detail can be detrimental.

On-policy self-distillation (OPSD) leverages a frozen copy of the base model as a teacher that scores the student's own rollouts under a privileged context. Conventionally, this privileged context is a complete reference solution that bundles the final answer with one particular reasoning path. The central tension explored in this work is that more privileged information does not always yield a better teacher. The authors hypothesize that the optimal level of abstraction in the privileged context depends on what the student can still act on. They systematically compare the default full-solution context against three offline-compiled abstractions—a named strategy, a method-independent framing, and a problem category—as well as an answer-only control that preserves the destination but removes the reasoning path. The study aims to identify how context design interacts with student scale and task type, and whether initial teacher-student KL divergence can predict downstream performance.

The experimental setup holds the student view and training procedure fixed within each model scale (4B and 8B parameters). For each scale, the authors evaluate five privileged context conditions: (1) full reference solution (default), (2) named strategy, (3) method-independent framing, (4) problem category, and (5) answer-only. The intermediate contexts are compiled offline, meaning they are generated without access to the student's rollouts. The teacher model is a frozen copy of the base model that scores the student's on-policy rollouts under the given context. Training uses standard OPSD objectives. The primary evaluation is on competition mathematics tasks, with in-domain peak mean accuracy as the key metric. Comparisons are conducted across three random seeds to assess robustness. The authors also measure the number of hint tokens stored for each context type and compute the initial teacher-student KL divergence to test its correlation with downstream performance. The experimental design isolates the effect of context abstraction while controlling for model scale and training dynamics.

Why it matters

The findings challenge the assumption that richer privileged information always benefits self-distillation. Instead, the level of abstraction must align with the student's capacity to act on it. Intermediate contexts like named strategies and method-independent framings provide sufficient guidance without overwhelming the student with a single reasoning path that may not match its own rollout distribution. The answer-only control's competitiveness suggests that for some scales and tasks, the destination alone is enough, and the path may introduce noise. The variation in preferred context across scales implies that as students grow more capable, they benefit from more abstract, method-independent hints. The lack of correlation between initial KL divergence and downstream performance indicates that static measures of teacher-student divergence are insufficient for predicting OPSD success. These insights have practical implications for designing efficient and effective self-distillation pipelines, particularly in reducing token storage costs while improving performance. Future work should explore adaptive context selection based on student scale and task characteristics.

Who should read this

CS practitioners and researchers

Opening member content…