BitCal-TTS: Bit-Calibrated Test-Time Scaling for Quantized Reasoning Models
Researchers have introduced BitCal-TTS, a novel lightweight runtime controller designed to enhance the performance of large reasoning models operating under post-training quantization. While quantization reduces memory and latency requirements, it often distorts online signals, leading to miscalibrated confidence and harmful early halting during inference. BitCal-TTS addresses this by combining inexpensive online proxies for token-level uncertainty, bit-conditioned confidence rescaling, and a bit-aware post-marker confirmation horizon. The method requires no fine-tuning of the base model and integrates seamlessly with standard Hugging Face 4-bit inference. Evaluations on GSM8K shards using Qwen2.5 Instruct models (7B and 14B scales) demonstrate significant improvements. At a token cap of 512, BitCal-TTS achieved accuracy gains of 3.7 points for the 7B model and 2.8 points for the 14B model compared to non-bit-aware baselines. Additionally, the premature-stop rate decreased substantially, falling from 14.8% to 11.1% for the 7B model and from 17.1% to 11.4% for the 14B model. The authors emphasize that while statistical power is limited due to partial-shard comparisons, the approach preserves substantial token savings relative to fixed-budget decoding. Code and scripts for full reproduction have been released to support further research and adoption.
Wire timeline
BitCal-TTS: Bit-Calibrated Test-Time Scaling for Quantized Reasoning Models
Researchers have introduced BitCal-TTS, a novel lightweight runtime controller designed to enhance the performance of large reasoning models operating under post-training quantization. While quantization reduces memory and latency requirements, it often distorts online signals, leading to miscalibrated confidence and harmful early halting during inference. BitCal-TTS addresses this by combining inexpensive online proxies for token-level uncertainty, bit-conditioned confidence rescaling, and a bit-aware post-marker confirmation horizon. The method requires no fine-tuning of the base model and integrates seamlessly with standard Hugging Face 4-bit inference. Evaluations on GSM8K shards using Qwen2.5 Instruct models (7B and 14B scales) demonstrate significant improvements. At a token cap of 512, BitCal-TTS achieved accuracy gains of 3.7 points for the 7B model and 2.8 points for the 14B model compared to non-bit-aware baselines. Additionally, the premature-stop rate decreased substantially, falling from 14.8% to 11.1% for the 7B model and from 17.1% to 11.4% for the 14B model. The authors emphasize that while statistical power is limited due to partial-shard comparisons, the approach preserves substantial token savings relative to fixed-budget decoding. Code and scripts for full reproduction have been released to support further research and adoption.
cs.AI updates on arXiv.org