Bridging Code and Knowledge with LLMs in Heuristic Design

A new research paper submitted to arXiv introduces a novel approach to Automatic Heuristic Design (AHD) for combinatorial optimization using Large Language Models (LLMs). The authors argue that existing methods, which rely on a bottom-up paradigm of searching executable programs and refining them based on execution feedback, are incomplete. They propose a complementary top-down perspective where knowledge serves as the primary search object, while code acts merely to instantiate and test this knowledge. This shift aims to make learned insights explicit and reusable across different problems and search trajectories. The study formalizes this approach through a statistical-learning view, highlighting a distortion-compression trade-off, and implements it within both population-based and tree-based AHD frameworks. Results indicate that this knowledge-first search strategy enhances discovery efficiency, transferability, and generalization, often outperforming traditional code-centric pipelines. Furthermore, combining both bottom-up and top-down strategies yields additional performance gains. The findings suggest that future progress in AHD relies on iteratively constructing and evolving interpretable hypotheses that retain value beyond single search instances, marking a significant advancement in leveraging LLMs for complex optimization tasks.