SWIFT: Prompt-Adaptive Memory for Efficient Interactive Long Video Generation
Researchers have introduced SWIFT, a novel training-free framework designed to enhance the efficiency and quality of interactive long-video generation using causal video diffusion models. Addressing the challenge of continuous semantic switching in streaming video, SWIFT resolves the mismatch between cached video history and prompt updates. Unlike current methods that rely on redundant cache rebuilding or fixed memory budgets, SWIFT employs a lightweight Semantic Injection Cache to augment existing memory rather than reconstructing it. The framework features head-wise semantic injection, ensuring attention heads receive prompt updates proportional to their alignment with the current video state. Additionally, an Adaptive Dynamic Window optimizes temporal memory allocation by adjusting context size based on prompt phases, while segment-level semantic anchors maintain long-range consistency. This approach significantly reduces inference costs and preserves temporal coherence. Benchmark results demonstrate that SWIFT achieves a generation speed of 22.6 FPS on a single H100 GPU, outperforming state-of-the-art methods in efficiency while maintaining high generation quality. The code for this innovative solution is publicly available, marking a substantial advancement in multi-prompt long-video generation technology.
Wire timeline
SWIFT: Prompt-Adaptive Memory for Efficient Interactive Long Video Generation
Researchers have introduced SWIFT, a novel training-free framework designed to enhance the efficiency and quality of interactive long-video generation using causal video diffusion models. Addressing the challenge of continuous semantic switching in streaming video, SWIFT resolves the mismatch between cached video history and prompt updates. Unlike current methods that rely on redundant cache rebuilding or fixed memory budgets, SWIFT employs a lightweight Semantic Injection Cache to augment existing memory rather than reconstructing it. The framework features head-wise semantic injection, ensuring attention heads receive prompt updates proportional to their alignment with the current video state. Additionally, an Adaptive Dynamic Window optimizes temporal memory allocation by adjusting context size based on prompt phases, while segment-level semantic anchors maintain long-range consistency. This approach significantly reduces inference costs and preserves temporal coherence. Benchmark results demonstrate that SWIFT achieves a generation speed of 22.6 FPS on a single H100 GPU, outperforming state-of-the-art methods in efficiency while maintaining high generation quality. The code for this innovative solution is publicly available, marking a substantial advancement in multi-prompt long-video generation technology.
cs.AI updates on arXiv.org