SenseBench: A Benchmark for Remote Sensing Low-Level Visual Perception in Large Vision-Language Models
Researchers have introduced SenseBench, the first dedicated diagnostic benchmark designed to evaluate low-level visual perception and description capabilities of Large Vision-Language Models (VLMs) in remote sensing (RS). Current image quality assessment methods often fail to provide interpretable diagnostics for physics-driven RS degradations, while existing VLMs are biased toward ground-level natural images. SenseBench addresses this domain gap with over 10,000 curated instances across 22 fine-grained degradation categories, utilizing a physics-based hierarchical taxonomy. The benchmark employs two evaluation protocols: objective visual perception and subjective diagnostic description. A comprehensive assessment of 29 state-of-the-art VLMs revealed significant challenges, including skewed domain priors, multi-distortion collapse, fluency illusions, and a perception-description inversion effect. This initiative aims to provide a robust testbed and high-quality data to advance VLM development for remote sensing applications. The associated code and datasets are publicly available to support further research in overcoming domain-specific visual perception limitations.
Wire timeline
SenseBench: A Benchmark for Remote Sensing Low-Level Visual Perception in Large Vision-Language Models
Researchers have introduced SenseBench, the first dedicated diagnostic benchmark designed to evaluate low-level visual perception and description capabilities of Large Vision-Language Models (VLMs) in remote sensing (RS). Current image quality assessment methods often fail to provide interpretable diagnostics for physics-driven RS degradations, while existing VLMs are biased toward ground-level natural images. SenseBench addresses this domain gap with over 10,000 curated instances across 22 fine-grained degradation categories, utilizing a physics-based hierarchical taxonomy. The benchmark employs two evaluation protocols: objective visual perception and subjective diagnostic description. A comprehensive assessment of 29 state-of-the-art VLMs revealed significant challenges, including skewed domain priors, multi-distortion collapse, fluency illusions, and a perception-description inversion effect. This initiative aims to provide a robust testbed and high-quality data to advance VLM development for remote sensing applications. The associated code and datasets are publicly available to support further research in overcoming domain-specific visual perception limitations.
cs.AI updates on arXiv.org