Lattice Deduction Transformers: A New Recurrent Model for Logically Sound Reasoning
Researchers have introduced the Lattice Deduction Transformer (LDT), a novel recurrent transformer architecture designed to approximate logically sound deduction. By projecting its latent state through a lattice between forward passes, the model mimics the deduction process of search-based constraint solvers. Training is conducted on-policy using a domain-agnostic, abstract-interpretation-based approximation of solution candidates. The results demonstrate significant efficiency and accuracy improvements over existing methods. An 800K-parameter LDT achieved 100% accuracy on Sudoku-Extreme and Snowflake Sudoku benchmarks, while a 1.8M-parameter variant reached 99.9% accuracy on Maze-Hard. Notably, the model remains empirically sound by either returning a correct answer or abstaining, avoiding incorrect outputs. In stark contrast, frontier Large Language Models (LLMs) scored 0% on these same benchmarks. This development highlights a potential shift towards more reliable, logic-driven AI systems for complex reasoning tasks, achieving high performance at a fraction of the training cost required by prior small recurrent reasoners.
Wire timeline
Lattice Deduction Transformers: A New Recurrent Model for Logically Sound Reasoning
Researchers have introduced the Lattice Deduction Transformer (LDT), a novel recurrent transformer architecture designed to approximate logically sound deduction. By projecting its latent state through a lattice between forward passes, the model mimics the deduction process of search-based constraint solvers. Training is conducted on-policy using a domain-agnostic, abstract-interpretation-based approximation of solution candidates. The results demonstrate significant efficiency and accuracy improvements over existing methods. An 800K-parameter LDT achieved 100% accuracy on Sudoku-Extreme and Snowflake Sudoku benchmarks, while a 1.8M-parameter variant reached 99.9% accuracy on Maze-Hard. Notably, the model remains empirically sound by either returning a correct answer or abstaining, avoiding incorrect outputs. In stark contrast, frontier Large Language Models (LLMs) scored 0% on these same benchmarks. This development highlights a potential shift towards more reliable, logic-driven AI systems for complex reasoning tasks, achieving high performance at a fraction of the training cost required by prior small recurrent reasoners.
cs.AI updates on arXiv.org