Causal Probing for Internal Visual Representations in Multimodal Large Language Models

Researchers have introduced a novel causal framework utilizing activation steering to investigate the internal visual representation mechanisms of Multimodal Large Language Models (MLLMs). The study addresses the limited understanding of how MLLMs encode and ground visual concepts. Through systematic interventions across four concept categories, the analysis reveals a significant divergence in encoding strategies: concrete entities are stored via localized memorization, while abstract concepts are distributed globally across the network. This finding identifies increasing model depth as a critical factor for handling complex abstract concepts, whereas entity localization remains stable regardless of scale. Additionally, reverse steering experiments uncover a compensatory mechanism between perception and generation, where blocking output triggers latent activation surges. The research further highlights a disconnect in visual reasoning, showing that while MLLMs recognize geometric relations, they treat them as static features rather than engaging in the procedural execution required for abstract problem-solving. These insights provide a deeper mechanistic understanding of scaling laws and internal processing limitations in current AI architectures.