SAFformer: Improving Spiking Transformer via Active Predictive Filtering
Researchers have introduced SAFformer, a novel Spiking Transformer architecture designed to enhance the efficiency and performance of Spiking Neural Networks (SNNs). While SNNs are promising for low-power applications due to their biological plausibility, existing models often suffer from a passive reactive paradigm that struggles with redundant visual data. SAFformer addresses this by implementing an active predictive filtering mechanism inspired by the brain's predictive coding. This approach actively suppresses predictable signals, allowing the model to focus on salient visual features. Extensive experiments demonstrate that SAFformer achieves state-of-the-art results on CIFAR-10, CIFAR-100, and CIFAR10-DVS datasets. Notably, on the ImageNet-1K benchmark, it attains an 80.50% Top-1 accuracy with only 26.58 million parameters and an energy consumption of 5.88 mJ. This breakthrough highlights an exceptional balance between computational accuracy and energy efficiency, marking a significant advancement in the development of low-power, biologically plausible artificial intelligence systems for computer vision tasks.
Wire timeline
SAFformer: Improving Spiking Transformer via Active Predictive Filtering
Researchers have introduced SAFformer, a novel Spiking Transformer architecture designed to enhance the efficiency and performance of Spiking Neural Networks (SNNs). While SNNs are promising for low-power applications due to their biological plausibility, existing models often suffer from a passive reactive paradigm that struggles with redundant visual data. SAFformer addresses this by implementing an active predictive filtering mechanism inspired by the brain's predictive coding. This approach actively suppresses predictable signals, allowing the model to focus on salient visual features. Extensive experiments demonstrate that SAFformer achieves state-of-the-art results on CIFAR-10, CIFAR-100, and CIFAR10-DVS datasets. Notably, on the ImageNet-1K benchmark, it attains an 80.50% Top-1 accuracy with only 26.58 million parameters and an energy consumption of 5.88 mJ. This breakthrough highlights an exceptional balance between computational accuracy and energy efficiency, marking a significant advancement in the development of low-power, biologically plausible artificial intelligence systems for computer vision tasks.
cs.AI updates on arXiv.org