Catch Your Breath: Adaptive Computation for Self-Paced Sequence Production

Researchers have introduced "Catch Your Breath" (CYB), a novel supervised loss function designed to enhance foundation models through adaptive computation. Unlike traditional width-based scaling methods that treat pause tokens as static barriers, CYB frames inference as a sequential-decision problem. This approach allows models to dynamically regulate their processing time by emitting special outputs to request additional compute steps, effectively signaling when they are ready to respond. The method enables models to abstain multiple times for longer delays, optimizing resource usage without increasing computational or memory costs. Experimental results indicate that CYB significantly outperforms standard cross-entropy objectives in both pretraining and fine-tuning scenarios. Key improvements include reduced perplexity and enhanced downstream accuracy, demonstrating the efficacy of allowing models to autonomously scale compute steps per input token. This development represents a significant advancement in inference-time scaling, offering a more flexible and efficient mechanism for managing model expressivity and response timing in artificial intelligence systems.