Recursive Language Models: A New Paradigm for Processing Long Contexts
Researchers have introduced Recursive Language Models (RLMs), a novel inference paradigm designed to enable large language models (LLMs) to process arbitrarily long prompts by treating them as an external environment. This approach allows LLMs to programmatically examine, decompose, and recursively call themselves on prompt snippets, effectively bypassing traditional context window limitations. The study demonstrates that RLMs can handle inputs up to two orders of magnitude larger than standard model contexts while maintaining comparable costs. Performance evaluations reveal that RLMs significantly outperform vanilla frontier LLMs and existing long-context scaffolds, such as CodeAct and Claude Code, across diverse tasks. Specifically, the post-trained RLM-Qwen3-8B model achieved a 28.3% average improvement over its base Qwen3-8B counterpart and approached the quality of GPT-5 in several long-context scenarios. This breakthrough suggests a scalable solution for enhancing LLM capabilities in handling extensive data without requiring massive architectural changes, offering substantial efficiency gains for complex computational tasks.
Wire timeline
Recursive Language Models: A New Paradigm for Processing Long Contexts
Researchers have introduced Recursive Language Models (RLMs), a novel inference paradigm designed to enable large language models (LLMs) to process arbitrarily long prompts by treating them as an external environment. This approach allows LLMs to programmatically examine, decompose, and recursively call themselves on prompt snippets, effectively bypassing traditional context window limitations. The study demonstrates that RLMs can handle inputs up to two orders of magnitude larger than standard model contexts while maintaining comparable costs. Performance evaluations reveal that RLMs significantly outperform vanilla frontier LLMs and existing long-context scaffolds, such as CodeAct and Claude Code, across diverse tasks. Specifically, the post-trained RLM-Qwen3-8B model achieved a 28.3% average improvement over its base Qwen3-8B counterpart and approached the quality of GPT-5 in several long-context scenarios. This breakthrough suggests a scalable solution for enhancing LLM capabilities in handling extensive data without requiring massive architectural changes, offering substantial efficiency gains for complex computational tasks.
cs.AI updates on arXiv.org