TIDES: Implicit Time-Awareness in Selective State Space Models
Researchers have introduced TIDES, a novel selective state space model (SSM) designed to enhance the modeling of irregular time series data. While existing selective SSMs like Mamba offer strong per-token expressivity by making the time discretization step input-dependent, they lose the physical meaning of sampling intervals. Conversely, continuous-time SSMs like S5 preserve physical timing but lack expressivity due to linear time-invariant dynamics. TIDES reconciles these approaches by shifting input dependence from the step size to the diagonal state matrix. This allows the model to retain the physical meaning of time steps while maintaining high expressivity. The architecture was validated using a new 'Fading Flash' benchmark, which tests input dependence and extrapolation capabilities. TIDES demonstrated superior performance, achieving state-of-the-art average ranks on large-scale benchmarks, including UEA time-series classification and Physiome-ODE regression. The study highlights a significant advancement in sequence modeling for scientific and irregular temporal data, with code made publicly available for further research and application.
Wire timeline
TIDES: Implicit Time-Awareness in Selective State Space Models
Researchers have introduced TIDES, a novel selective state space model (SSM) designed to enhance the modeling of irregular time series data. While existing selective SSMs like Mamba offer strong per-token expressivity by making the time discretization step input-dependent, they lose the physical meaning of sampling intervals. Conversely, continuous-time SSMs like S5 preserve physical timing but lack expressivity due to linear time-invariant dynamics. TIDES reconciles these approaches by shifting input dependence from the step size to the diagonal state matrix. This allows the model to retain the physical meaning of time steps while maintaining high expressivity. The architecture was validated using a new 'Fading Flash' benchmark, which tests input dependence and extrapolation capabilities. TIDES demonstrated superior performance, achieving state-of-the-art average ranks on large-scale benchmarks, including UEA time-series classification and Physiome-ODE regression. The study highlights a significant advancement in sequence modeling for scientific and irregular temporal data, with code made publicly available for further research and application.
cs.AI updates on arXiv.org