DeepRefine: Agent-Compiled Knowledge Refinement via Reinforcement Learning
Researchers have introduced DeepRefine, a novel large language model (LLM)-based reasoning system designed to enhance the quality of agent-compiled knowledge bases. These databases, crucial for LLM agents in complex tasks, often suffer from incompleteness, incorrectness, and redundancy, which degrade performance over time. DeepRefine addresses these issues by engaging in multi-turn interactions with the knowledge base to perform abductive diagnosis on interaction history. It identifies likely defects and executes targeted refinement actions for incremental updates. A key innovation is the Gain-Beyond-Draft (GBD) reward mechanism, which allows the model to optimize refinement policies through end-to-end reinforcement learning without requiring gold standard references. Extensive experiments indicate that DeepRefine consistently outperforms strong baselines, delivering significant improvements in downstream task performance. This development represents a significant step forward in maintaining high-fidelity external knowledge for AI agents, ensuring more reliable and accurate information retrieval in open-ended, knowledge-intensive applications.
Wire timeline
DeepRefine: Agent-Compiled Knowledge Refinement via Reinforcement Learning
Researchers have introduced DeepRefine, a novel large language model (LLM)-based reasoning system designed to enhance the quality of agent-compiled knowledge bases. These databases, crucial for LLM agents in complex tasks, often suffer from incompleteness, incorrectness, and redundancy, which degrade performance over time. DeepRefine addresses these issues by engaging in multi-turn interactions with the knowledge base to perform abductive diagnosis on interaction history. It identifies likely defects and executes targeted refinement actions for incremental updates. A key innovation is the Gain-Beyond-Draft (GBD) reward mechanism, which allows the model to optimize refinement policies through end-to-end reinforcement learning without requiring gold standard references. Extensive experiments indicate that DeepRefine consistently outperforms strong baselines, delivering significant improvements in downstream task performance. This development represents a significant step forward in maintaining high-fidelity external knowledge for AI agents, ensuring more reliable and accurate information retrieval in open-ended, knowledge-intensive applications.
cs.AI updates on arXiv.org