The Trap of Trajectory: Mitigating Spurious Correlations in Agentic Memory

A new research paper titled "The Trap of Trajectory" addresses a critical vulnerability in agentic memory systems for Large Language Models (LLMs). While agentic memory allows LLMs to retain and reuse information across contexts, it introduces risks of spurious correlations, where retrieved data contains misleading evidence that propagates errors into downstream decisions. The authors benchmark canonical spurious patterns, revealing that while memory aids reasoning on clean inputs, it amplifies reliance on these erroneous patterns when present. To counter this, the study proposes CAMEL, a plug-and-play calibration method applicable to diverse memory architectures during both write and retrieval phases. CAMEL effectively reduces dependence on spurious correlations without compromising performance on valid inputs and remains robust against adaptive attacks. This lightweight solution aims to enhance the reliability and safety of deploying agentic memory in AI systems, offering a principled approach to mitigating reasoning errors caused by miscorrelated historical data.