Dropbox Uses LLMs to Amplify Human Labeling for Dash Search Relevance
Dropbox has detailed its approach to training the search ranking models for Dash, its AI-powered enterprise search tool. The system utilizes a retrieval-augmented generation (RAG) pattern, where an AI retrieves relevant company-specific documents and generates grounded responses. To ensure high-quality results, Dropbox employs a hybrid labeling strategy combining human judgment with Large Language Model (LLM) assistance. While traditional human labeling provides accurate relevance scores on a 1-5 scale, it is expensive and difficult to scale. Conversely, relying solely on user behavior signals like clicks can be biased and incomplete. Dropbox addresses these challenges by starting with a small set of internal, human-labeled data to establish quality benchmarks. They then amplify this effort using LLMs to generate relevance labels at scale. This method allows the XGBoost-based ranking model to learn effectively from extensive examples of query-document pairs, improving its ability to prioritize useful documents. The article highlights how this mix of human expertise and AI scalability enhances the precision of search results within large enterprises, ensuring that Dash delivers accurate and contextually relevant answers to user queries.
Wire timeline
Dropbox Uses LLMs to Amplify Human Labeling for Dash Search Relevance
Dropbox has detailed its approach to training the search ranking models for Dash, its AI-powered enterprise search tool. The system utilizes a retrieval-augmented generation (RAG) pattern, where an AI retrieves relevant company-specific documents and generates grounded responses. To ensure high-quality results, Dropbox employs a hybrid labeling strategy combining human judgment with Large Language Model (LLM) assistance. While traditional human labeling provides accurate relevance scores on a 1-5 scale, it is expensive and difficult to scale. Conversely, relying solely on user behavior signals like clicks can be biased and incomplete. Dropbox addresses these challenges by starting with a small set of internal, human-labeled data to establish quality benchmarks. They then amplify this effort using LLMs to generate relevance labels at scale. This method allows the XGBoost-based ranking model to learn effectively from extensive examples of query-document pairs, improving its ability to prioritize useful documents. The article highlights how this mix of human expertise and AI scalability enhances the precision of search results within large enterprises, ensuring that Dash delivers accurate and contextually relevant answers to user queries.
Dropbox Tech Blog