Reasoning Compression with Mixed-Policy Distillation
Researchers have introduced Mixed-Policy Distillation (MPD), a novel framework designed to enhance the efficiency of reasoning-centric large language models (LLMs). While larger LLMs achieve strong performance through intermediate reasoning trajectories, they often incur high token usage and inference costs. Conversely, smaller models, preferred for real-world deployment due to memory and latency constraints, tend to generate redundant and verbose reasoning traces. MPD addresses this by transferring concise reasoning behaviors from larger teacher models to smaller student models. Unlike traditional on-policy or off-policy distillation methods, MPD combines their strengths by having the teacher rewrite student-sampled trajectories into more concise forms. The student model is then trained via KL-based alignment on these compressed trajectories. This approach preserves the student's policy exploration while injecting teacher-guided compression. Experimental results using Qwen3-1.7B demonstrate that MPD reduces token usage by up to 27.1% while simultaneously improving performance across multiple reasoning benchmarks. This development offers a significant advancement in making small-model reasoning more efficient and cost-effective for practical applications.
Wire timeline
Reasoning Compression with Mixed-Policy Distillation
Researchers have introduced Mixed-Policy Distillation (MPD), a novel framework designed to enhance the efficiency of reasoning-centric large language models (LLMs). While larger LLMs achieve strong performance through intermediate reasoning trajectories, they often incur high token usage and inference costs. Conversely, smaller models, preferred for real-world deployment due to memory and latency constraints, tend to generate redundant and verbose reasoning traces. MPD addresses this by transferring concise reasoning behaviors from larger teacher models to smaller student models. Unlike traditional on-policy or off-policy distillation methods, MPD combines their strengths by having the teacher rewrite student-sampled trajectories into more concise forms. The student model is then trained via KL-based alignment on these compressed trajectories. This approach preserves the student's policy exploration while injecting teacher-guided compression. Experimental results using Qwen3-1.7B demonstrate that MPD reduces token usage by up to 27.1% while simultaneously improving performance across multiple reasoning benchmarks. This development offers a significant advancement in making small-model reasoning more efficient and cost-effective for practical applications.
cs.AI updates on arXiv.org