PARD-2: Target-Aligned Parallel Draft Model for Dual-Mode Speculative Decoding
Researchers have introduced PARD-2, a novel dual-mode speculative decoding framework designed to accelerate Large Language Model (LLM) inference. Addressing the misalignment between existing draft model training objectives and the goal of maximizing consecutive token acceptance, PARD-2 reformulates the optimization objective to focus on overall acceptance length rather than just token prediction accuracy. The framework incorporates Confidence-Adaptive Token (CAT) optimization, which adaptively reweights tokens to better align with the verification process. A key innovation is its ability to support both target-dependent and target-independent modes using a single draft model. Experimental results demonstrate significant performance improvements, with PARD-2 achieving up to 6.94x lossless acceleration. On the Llama3.1-8B model, it surpasses previous state-of-the-art methods like EAGLE-3 by 1.9x and its predecessor PARD by 1.3x. This advancement highlights a critical step in optimizing LLM efficiency for diverse tasks and models. The associated code has been made publicly available via GitHub, facilitating further research and application in the field of artificial intelligence and computational linguistics.
Wire timeline
PARD-2: Target-Aligned Parallel Draft Model for Dual-Mode Speculative Decoding
Researchers have introduced PARD-2, a novel dual-mode speculative decoding framework designed to accelerate Large Language Model (LLM) inference. Addressing the misalignment between existing draft model training objectives and the goal of maximizing consecutive token acceptance, PARD-2 reformulates the optimization objective to focus on overall acceptance length rather than just token prediction accuracy. The framework incorporates Confidence-Adaptive Token (CAT) optimization, which adaptively reweights tokens to better align with the verification process. A key innovation is its ability to support both target-dependent and target-independent modes using a single draft model. Experimental results demonstrate significant performance improvements, with PARD-2 achieving up to 6.94x lossless acceleration. On the Llama3.1-8B model, it surpasses previous state-of-the-art methods like EAGLE-3 by 1.9x and its predecessor PARD by 1.3x. This advancement highlights a critical step in optimizing LLM efficiency for diverse tasks and models. The associated code has been made publicly available via GitHub, facilitating further research and application in the field of artificial intelligence and computational linguistics.
cs.AI updates on arXiv.org