CoreQ: Learning-Free Mismatch Correction and Successive Rounding for Quantization
Researchers have introduced CoreQ, a novel learning-free Post-Training Quantization (PTQ) framework designed to enhance the efficient deployment of Large Language Models (LLMs). PTQ typically maps pretrained weights to low-bit formats without retraining, but sequential quantization often causes mismatch errors where earlier layer inaccuracies distort inputs for subsequent layers. Existing methods struggle with balancing mismatch correction, risking overfitting or ignoring layer-specific reliability. CoreQ addresses this by applying a closed-form coefficient for mismatch correction, derived from geometric decomposition. This approach adapts corrections across layers, reduces overfitting to limited calibration data, and eliminates the need for hyperparameter tuning. The framework minimizes the resulting triangular least-squares objective using an efficient greedy successive-rounding solver, alongside K-CoreQ, a bounded beam-search extension that offers improved performance at a modest computational cost. Evaluations across various LLM families, scales, and bit-widths demonstrate that CoreQ significantly improves perplexity and downstream accuracy compared to strong PTQ baselines. This advancement provides a robust, tuning-free solution for optimizing LLM efficiency, marking a significant step in making high-performance AI models more accessible and deployable on resource-constrained hardware.
Wire timeline
CoreQ: Learning-Free Mismatch Correction and Successive Rounding for Quantization
Researchers have introduced CoreQ, a novel learning-free Post-Training Quantization (PTQ) framework designed to enhance the efficient deployment of Large Language Models (LLMs). PTQ typically maps pretrained weights to low-bit formats without retraining, but sequential quantization often causes mismatch errors where earlier layer inaccuracies distort inputs for subsequent layers. Existing methods struggle with balancing mismatch correction, risking overfitting or ignoring layer-specific reliability. CoreQ addresses this by applying a closed-form coefficient for mismatch correction, derived from geometric decomposition. This approach adapts corrections across layers, reduces overfitting to limited calibration data, and eliminates the need for hyperparameter tuning. The framework minimizes the resulting triangular least-squares objective using an efficient greedy successive-rounding solver, alongside K-CoreQ, a bounded beam-search extension that offers improved performance at a modest computational cost. Evaluations across various LLM families, scales, and bit-widths demonstrate that CoreQ significantly improves perplexity and downstream accuracy compared to strong PTQ baselines. This advancement provides a robust, tuning-free solution for optimizing LLM efficiency, marking a significant step in making high-performance AI models more accessible and deployable on resource-constrained hardware.
cs.AI updates on arXiv.org