Primal-Dual Guided Decoding for Constrained Discrete Diffusion

Researchers have introduced a novel inference-time method called primal-dual guided decoding to address the challenge of enforcing global property constraints in discrete diffusion models. These models generate structured sequences by progressively unmasking tokens, but ensuring specific constraints during this process has remained difficult. The proposed technique formulates constrained generation as a KL-regularized optimization problem, solving it online using adaptive Lagrangian multipliers. At each denoising step, the method adjusts token logits with an additive, constraint-dependent bias, updated via mirror descent based on constraint violations. This approach ensures the constrained distribution stays close to the model's original unconstrained distribution while satisfying required conditions. Key advantages include no need for retraining, no additional model evaluations beyond standard sampling, support for multiple simultaneous constraints, and formal bounds on constraint violation. The method was evaluated across diverse applications, including topical text generation, molecular design, and music playlist generation. Results demonstrated that the algorithm significantly improves constraint satisfaction while maintaining high domain-specific quality metrics, offering a versatile solution for controlled generative AI tasks without requiring model modification.