MedThink: Enhancing Diagnostic Accuracy in Small Models via Teacher-Guided Reasoning Correction
Researchers have introduced MedThink, a novel two-stage distillation framework designed to improve the diagnostic accuracy of small language models (SLMs) in clinical settings. While large language models offer strong reasoning capabilities, their high computational costs limit deployment in resource-constrained environments. Traditional knowledge distillation often fails to preserve the structured reasoning necessary for reliable medical diagnosis. MedThink addresses this by first using a teacher LLM to inject domain-knowledge explanations into the student model, establishing a foundational knowledge base. In the second stage, the teacher evaluates student errors and generates detailed reasoning chains to refine the model's diagnostic logic through further fine-tuning. Evaluations on general medical benchmarks and a specialized gastroenterology dataset showed that MedThink outperforms six existing distillation strategies. It achieved up to a 12.7% improvement over baseline student models in general tasks and reached a top accuracy of 56.4% in gastroenterology assessments. This approach demonstrates that iterative, reasoning-centered distillation can significantly enhance both accuracy and generalization in SLMs while maintaining computational efficiency. The associated code and data are publicly available to support further research and application in medical AI.
Wire timeline
MedThink: Enhancing Diagnostic Accuracy in Small Models via Teacher-Guided Reasoning Correction
Researchers have introduced MedThink, a novel two-stage distillation framework designed to improve the diagnostic accuracy of small language models (SLMs) in clinical settings. While large language models offer strong reasoning capabilities, their high computational costs limit deployment in resource-constrained environments. Traditional knowledge distillation often fails to preserve the structured reasoning necessary for reliable medical diagnosis. MedThink addresses this by first using a teacher LLM to inject domain-knowledge explanations into the student model, establishing a foundational knowledge base. In the second stage, the teacher evaluates student errors and generates detailed reasoning chains to refine the model's diagnostic logic through further fine-tuning. Evaluations on general medical benchmarks and a specialized gastroenterology dataset showed that MedThink outperforms six existing distillation strategies. It achieved up to a 12.7% improvement over baseline student models in general tasks and reached a top accuracy of 56.4% in gastroenterology assessments. This approach demonstrates that iterative, reasoning-centered distillation can significantly enhance both accuracy and generalization in SLMs while maintaining computational efficiency. The associated code and data are publicly available to support further research and application in medical AI.
cs.AI updates on arXiv.org