CoCoDA: Co-evolving Compositional DAG for Tool-Augmented Agents
Researchers have introduced CoCoDA, a novel framework designed to enhance tool-augmented language models by addressing the challenges of scaling tool libraries. Traditional methods often treat tools as flat memories, leading to increased prompt costs and obscured code structures as libraries grow. CoCoDA proposes a co-evolutionary approach using a compositional code Directed Acyclic Graph (DAG), where nodes represent tools and edges encode dependencies. This structure allows for efficient retrieval through typed signature unification and behavioral filtering, keeping context usage within budget. During training, successful trajectories are converted into composite tools, and the planner is updated with rewards based on primitive expansion. Theoretical analysis confirms reduced retrieval costs and sublinear retrieval time. Empirical benchmarks in mathematical reasoning, tabular analysis, and coding tasks demonstrate that an 8B parameter student model using CoCoDA can match or exceed the performance of a 32B parameter teacher model on GSM8K and MATH datasets, significantly outperforming existing baselines.
Wire timeline
CoCoDA: Co-evolving Compositional DAG for Tool-Augmented Agents
Researchers have introduced CoCoDA, a novel framework designed to enhance tool-augmented language models by addressing the challenges of scaling tool libraries. Traditional methods often treat tools as flat memories, leading to increased prompt costs and obscured code structures as libraries grow. CoCoDA proposes a co-evolutionary approach using a compositional code Directed Acyclic Graph (DAG), where nodes represent tools and edges encode dependencies. This structure allows for efficient retrieval through typed signature unification and behavioral filtering, keeping context usage within budget. During training, successful trajectories are converted into composite tools, and the planner is updated with rewards based on primitive expansion. Theoretical analysis confirms reduced retrieval costs and sublinear retrieval time. Empirical benchmarks in mathematical reasoning, tabular analysis, and coding tasks demonstrate that an 8B parameter student model using CoCoDA can match or exceed the performance of a 32B parameter teacher model on GSM8K and MATH datasets, significantly outperforming existing baselines.
cs.AI updates on arXiv.org