Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction
A new research paper submitted to arXiv introduces Direct Corpus Interaction (DCI), a novel approach to information retrieval for agentic search tasks. Traditional retrieval systems rely on fixed similarity interfaces that compress access into a single top-k step, creating bottlenecks for complex agent workflows requiring multi-step reasoning and exact lexical constraints. DCI bypasses embedding models and vector indexes, allowing agents to search raw corpora directly using general-purpose terminal tools like grep and shell commands. This method eliminates the need for offline indexing and adapts seamlessly to evolving local data. Experimental results demonstrate that DCI substantially outperforms strong sparse, dense, and reranking baselines on BRIGHT and BEIR datasets. It also achieves high accuracy on BrowseComp-Plus and multi-hop QA tasks without conventional semantic retrievers. The study suggests that as language agents advance, retrieval quality depends significantly on the interface resolution between the model and the corpus, opening new design possibilities for agentic search systems.
Wire timeline
Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction
A new research paper submitted to arXiv introduces Direct Corpus Interaction (DCI), a novel approach to information retrieval for agentic search tasks. Traditional retrieval systems rely on fixed similarity interfaces that compress access into a single top-k step, creating bottlenecks for complex agent workflows requiring multi-step reasoning and exact lexical constraints. DCI bypasses embedding models and vector indexes, allowing agents to search raw corpora directly using general-purpose terminal tools like grep and shell commands. This method eliminates the need for offline indexing and adapts seamlessly to evolving local data. Experimental results demonstrate that DCI substantially outperforms strong sparse, dense, and reranking baselines on BRIGHT and BEIR datasets. It also achieves high accuracy on BrowseComp-Plus and multi-hop QA tasks without conventional semantic retrievers. The study suggests that as language agents advance, retrieval quality depends significantly on the interface resolution between the model and the corpus, opening new design possibilities for agentic search systems.
cs.AI updates on arXiv.org