Why Retrying Fails: Context Contamination in LLM Agent Pipelines
A new academic paper introduces the Context-Contaminated Restart Model (CCRM) to explain why Large Language Model (LLM) agents often fail when retrying multi-step tasks. The study highlights that failed attempts remain in the context window, contaminating subsequent retries and significantly elevating error rates compared to initial attempts. The authors derive five key theoretical results, including formulas for success probability, cascade overhead, and optimal budget allocation for pipeline depth. Empirical validation using SWE-bench Verified data reveals that standard independent models overestimate success rates by 17.4 percentage points, whereas CCRM fits the data with minimal error. The findings indicate a cascade ratio where contaminated error rates are 7.1 times higher than base rates. This research provides a formal treatment for a widely observed but previously unquantified phenomenon, offering strategies like clean-restart mechanisms to improve agent reliability. The work combines theoretical proofs with Monte Carlo experiments to confirm predictions, aiming to optimize resource allocation in AI agent pipelines.
Wire timeline
Why Retrying Fails: Context Contamination in LLM Agent Pipelines
A new academic paper introduces the Context-Contaminated Restart Model (CCRM) to explain why Large Language Model (LLM) agents often fail when retrying multi-step tasks. The study highlights that failed attempts remain in the context window, contaminating subsequent retries and significantly elevating error rates compared to initial attempts. The authors derive five key theoretical results, including formulas for success probability, cascade overhead, and optimal budget allocation for pipeline depth. Empirical validation using SWE-bench Verified data reveals that standard independent models overestimate success rates by 17.4 percentage points, whereas CCRM fits the data with minimal error. The findings indicate a cascade ratio where contaminated error rates are 7.1 times higher than base rates. This research provides a formal treatment for a widely observed but previously unquantified phenomenon, offering strategies like clean-restart mechanisms to improve agent reliability. The work combines theoretical proofs with Monte Carlo experiments to confirm predictions, aiming to optimize resource allocation in AI agent pipelines.
cs.AI updates on arXiv.org