Theorem-SFT: Enhancing AI Reasoning Generalization by Memorizing Theorems
A new research paper titled 'Memorize Theorems, Not Instances' addresses the limitations of Supervised Fine-Tuning (SFT) in artificial intelligence. While SFT is standard for task adaptation, it often harms reasoning generalization by causing models to memorize spurious surface correlations rather than underlying logic. The authors propose 'Theorem-SFT,' a method that shifts supervision toward explicit theorem application, teaching models how rules are invoked instead of just what answers look like. This approach yields significant performance improvements, including an 8.8% gain on the MATH benchmark for LLaMA3.2-3B-Instruct and a 20.27% increase on GeoQA for Qwen2.5-VL-7B-Instruct, without requiring modality-specific re-training. Furthermore, the study finds that fine-tuning only MLP layers matches full-layer performance, suggesting feed-forward components are central to reasoning rules. These findings reframe the debate on generalization failures, attributing them to memorizing incorrect inductive targets rather than memorization itself. The research highlights a pivotal shift in optimizing large language models for robust mathematical and logical reasoning capabilities.
Wire timeline
Theorem-SFT: Enhancing AI Reasoning Generalization by Memorizing Theorems
A new research paper titled 'Memorize Theorems, Not Instances' addresses the limitations of Supervised Fine-Tuning (SFT) in artificial intelligence. While SFT is standard for task adaptation, it often harms reasoning generalization by causing models to memorize spurious surface correlations rather than underlying logic. The authors propose 'Theorem-SFT,' a method that shifts supervision toward explicit theorem application, teaching models how rules are invoked instead of just what answers look like. This approach yields significant performance improvements, including an 8.8% gain on the MATH benchmark for LLaMA3.2-3B-Instruct and a 20.27% increase on GeoQA for Qwen2.5-VL-7B-Instruct, without requiring modality-specific re-training. Furthermore, the study finds that fine-tuning only MLP layers matches full-layer performance, suggesting feed-forward components are central to reasoning rules. These findings reframe the debate on generalization failures, attributing them to memorizing incorrect inductive targets rather than memorization itself. The research highlights a pivotal shift in optimizing large language models for robust mathematical and logical reasoning capabilities.
cs.AI updates on arXiv.org