cuRegOT: A GPU-Accelerated Solver for Entropic-Regularized Optimal Transport
Researchers have introduced cuRegOT, a high-performance GPU solver designed to address the computational bottlenecks of entropic-regularized Optimal Transport (OT) in large-scale machine learning applications. While OT is fundamental to modern AI, existing methods like the Sinkhorn algorithm suffer from slow convergence, and quasi-Newton methods face efficiency issues on GPUs due to serial symbolic analysis and irregular memory access. cuRegOT bridges this gap through specific algorithmic and architectural optimizations, including an amortized symbolic analysis strategy to reduce CPU bottlenecks, an asynchronous mechanism for generating Sinkhorn iterates, and a fused kernel for bandwidth-efficient gradient evaluation. These innovations are supported by rigorous theoretical guarantees ensuring algorithmic convergence. Extensive numerical experiments indicate that cuRegOT achieves significant speedups compared to state-of-the-art GPU-based solvers across various benchmark tasks. This development promises to enhance the efficiency and scalability of OT computations, facilitating broader adoption in complex machine learning models where computational cost has previously been a limiting factor.
Wire timeline
cuRegOT: A GPU-Accelerated Solver for Entropic-Regularized Optimal Transport
Researchers have introduced cuRegOT, a high-performance GPU solver designed to address the computational bottlenecks of entropic-regularized Optimal Transport (OT) in large-scale machine learning applications. While OT is fundamental to modern AI, existing methods like the Sinkhorn algorithm suffer from slow convergence, and quasi-Newton methods face efficiency issues on GPUs due to serial symbolic analysis and irregular memory access. cuRegOT bridges this gap through specific algorithmic and architectural optimizations, including an amortized symbolic analysis strategy to reduce CPU bottlenecks, an asynchronous mechanism for generating Sinkhorn iterates, and a fused kernel for bandwidth-efficient gradient evaluation. These innovations are supported by rigorous theoretical guarantees ensuring algorithmic convergence. Extensive numerical experiments indicate that cuRegOT achieves significant speedups compared to state-of-the-art GPU-based solvers across various benchmark tasks. This development promises to enhance the efficiency and scalability of OT computations, facilitating broader adoption in complex machine learning models where computational cost has previously been a limiting factor.
cs.AI updates on arXiv.org