GenCellAgent: Training-Free Cellular Image Segmentation via LLM Agents
Researchers have introduced GenCellAgent, a novel training-free multi-agent framework designed to address challenges in cellular image segmentation, such as heterogeneous modalities and limited annotations. The system orchestrates specialist segmenters and generalist vision-language models through a planner-executor-evaluator loop with long-term memory. Key capabilities include automatic tool routing, on-the-fly adaptation using reference images, and text-guided segmentation for organelles not covered by existing models. It also incorporates expert edits into memory to enable self-evolution and personalized workflows. Evaluations across seven benchmarks involving 4,718 images demonstrate that GenCellAgent consistently matches or exceeds the performance of individual tools and outperforms baselines in overall accuracy. Notably, it excels in out-of-distribution scenarios, recovering structures that dedicated tools miss, and supports segmentation of novel objects like the Golgi apparatus through iterative refinement. This approach offers a robust, adaptable solution for quantitative biology without the need for retraining, significantly reducing annotation burdens while aligning with user preferences.
Wire timeline
GenCellAgent: Training-Free Cellular Image Segmentation via LLM Agents
Researchers have introduced GenCellAgent, a novel training-free multi-agent framework designed to address challenges in cellular image segmentation, such as heterogeneous modalities and limited annotations. The system orchestrates specialist segmenters and generalist vision-language models through a planner-executor-evaluator loop with long-term memory. Key capabilities include automatic tool routing, on-the-fly adaptation using reference images, and text-guided segmentation for organelles not covered by existing models. It also incorporates expert edits into memory to enable self-evolution and personalized workflows. Evaluations across seven benchmarks involving 4,718 images demonstrate that GenCellAgent consistently matches or exceeds the performance of individual tools and outperforms baselines in overall accuracy. Notably, it excels in out-of-distribution scenarios, recovering structures that dedicated tools miss, and supports segmentation of novel objects like the Golgi apparatus through iterative refinement. This approach offers a robust, adaptable solution for quantitative biology without the need for retraining, significantly reducing annotation burdens while aligning with user preferences.
cs.AI updates on arXiv.org