COPE Framework Enables Cost-Efficient LLM Collaboration via Planning

Researchers have introduced COPE, a novel test-time collaboration framework designed to optimize the performance and cost efficiency of Large Language Models (LLMs). Addressing the trade-off between the high inference costs of large proprietary models and the limited capabilities of smaller, locally deployable models, COPE facilitates efficient collaboration between them. The framework employs a planner model to generate lightweight intermediate plans that guide a downstream executor model. Small and large models alternate roles as planner and executor in a multi-stage cascade, allowing them to combine their complementary strengths. Comprehensive experiments across diverse benchmarks, including mathematical reasoning, code generation, open-ended tasks, and agent tasks, demonstrate that COPE achieves performance comparable to large proprietary models. Crucially, this approach drastically reduces inference API costs. The study highlights planning as an effective prior for cost-efficient inference, offering a practical solution for applications requiring frequent LLM usage without prohibitive expenses. This development represents a significant step toward making advanced AI capabilities more accessible and economically viable for broader deployment.