Ace-Skill: Bootstrapping Multimodal Agents with Prioritized and Clustered Evolution
Researchers have introduced Ace-Skill, a novel co-evolutionary framework designed to enhance self-evolving multimodal agents by addressing data inefficiency and knowledge interference. Current self-evolving systems often suffer from a failure loop where uninformative rollouts generate noisy knowledge, degrading subsequent performance. Ace-Skill resolves this by combining a prioritized sampler with lazy-decay proficiency tracking to focus on informative, insufficiently mastered samples, alongside a clustered organizer for semantic knowledge grouping. This approach creates a virtuous cycle of high-quality knowledge acquisition. Tested across four multimodal tool-use benchmarks, Ace-Skill achieved a 35.46% relative improvement in Avg@4 accuracy. Notably, it enables an open-source 35B Mixture-of-Experts model to match or surpass proprietary counterparts. Furthermore, the acquired knowledge transfers effectively in a zero-shot manner to smaller 9B and 4B models, allowing resource-constrained agents to inherit advanced capabilities without additional training. The code has been made publicly available to support further research and development in efficient agent evolution.
Wire timeline
Ace-Skill: Bootstrapping Multimodal Agents with Prioritized and Clustered Evolution
Researchers have introduced Ace-Skill, a novel co-evolutionary framework designed to enhance self-evolving multimodal agents by addressing data inefficiency and knowledge interference. Current self-evolving systems often suffer from a failure loop where uninformative rollouts generate noisy knowledge, degrading subsequent performance. Ace-Skill resolves this by combining a prioritized sampler with lazy-decay proficiency tracking to focus on informative, insufficiently mastered samples, alongside a clustered organizer for semantic knowledge grouping. This approach creates a virtuous cycle of high-quality knowledge acquisition. Tested across four multimodal tool-use benchmarks, Ace-Skill achieved a 35.46% relative improvement in Avg@4 accuracy. Notably, it enables an open-source 35B Mixture-of-Experts model to match or surpass proprietary counterparts. Furthermore, the acquired knowledge transfers effectively in a zero-shot manner to smaller 9B and 4B models, allowing resource-constrained agents to inherit advanced capabilities without additional training. The code has been made publicly available to support further research and development in efficient agent evolution.
cs.AI updates on arXiv.org