Rethinking Agentic Search with Pi-Serini: Is Lexical Retrieval Sufficient?
Researchers from the University of Waterloo have introduced Pi-Serini, a new search agent designed to evaluate the efficacy of lexical retrieval methods in agentic search systems powered by advanced large language models (LLMs). The study challenges the prevailing assumption that dense retrievers are necessary for deep research tasks. By pairing the traditional BM25 lexical retriever with frontier LLMs like GPT-5.5, the team demonstrated that a well-configured lexical system with sufficient retrieval depth can outperform existing agents using dense retrievers. On the BrowseComp-Plus benchmark, Pi-Serini achieved an 83.1% answer accuracy and 94.7% surfaced evidence recall. Controlled ablations revealed that tuning BM25 improved accuracy by 18.0% and recall by 11.1%, while increasing retrieval depth boosted recall by an additional 25.3%. These findings suggest that lexical retrieval remains a viable and effective component for deep research systems when integrated with capable LLMs. The source code for Pi-Serini has been made publicly available to support further research in information retrieval and AI agent development.
Wire timeline
Rethinking Agentic Search with Pi-Serini: Is Lexical Retrieval Sufficient?
Researchers from the University of Waterloo have introduced Pi-Serini, a new search agent designed to evaluate the efficacy of lexical retrieval methods in agentic search systems powered by advanced large language models (LLMs). The study challenges the prevailing assumption that dense retrievers are necessary for deep research tasks. By pairing the traditional BM25 lexical retriever with frontier LLMs like GPT-5.5, the team demonstrated that a well-configured lexical system with sufficient retrieval depth can outperform existing agents using dense retrievers. On the BrowseComp-Plus benchmark, Pi-Serini achieved an 83.1% answer accuracy and 94.7% surfaced evidence recall. Controlled ablations revealed that tuning BM25 improved accuracy by 18.0% and recall by 11.1%, while increasing retrieval depth boosted recall by an additional 25.3%. These findings suggest that lexical retrieval remains a viable and effective component for deep research systems when integrated with capable LLMs. The source code for Pi-Serini has been made publicly available to support further research in information retrieval and AI agent development.
cs.AI updates on arXiv.org