CrossCult-KIBench: A Benchmark for Cross-Cultural Knowledge Insertion in MLLMs
Researchers have introduced CrossCult-KIBench, a new evaluation benchmark designed to address cultural biases in Multimodal Large Language Models (MLLMs). Since most MLLMs are trained on English-centric data, they often produce culturally inappropriate responses in non-Western contexts. This study defines the task of cross-cultural knowledge insertion, aiming to adapt models to specific cultural settings while preserving their original behavior elsewhere. The benchmark comprises 9,800 image-grounded cases across 49 visual scenarios, covering English, Chinese, and Arabic language-culture groups. It supports both single and sequential insertion evaluations. Additionally, the authors propose Memory-Conditioned Knowledge Insertion (MCKI) as a baseline method, which retrieves relevant cultural knowledge from external memory using frozen MLLM representations. Experimental results indicate that current methods struggle to balance effective cultural adaptation with behavioral preservation, highlighting significant challenges in developing culturally aware AI. This work underscores the need for more responsible and adaptive MLLMs, providing a crucial resource for future research in mitigating cultural misalignment in artificial intelligence systems.
Wire timeline
CrossCult-KIBench: A Benchmark for Cross-Cultural Knowledge Insertion in MLLMs
Researchers have introduced CrossCult-KIBench, a new evaluation benchmark designed to address cultural biases in Multimodal Large Language Models (MLLMs). Since most MLLMs are trained on English-centric data, they often produce culturally inappropriate responses in non-Western contexts. This study defines the task of cross-cultural knowledge insertion, aiming to adapt models to specific cultural settings while preserving their original behavior elsewhere. The benchmark comprises 9,800 image-grounded cases across 49 visual scenarios, covering English, Chinese, and Arabic language-culture groups. It supports both single and sequential insertion evaluations. Additionally, the authors propose Memory-Conditioned Knowledge Insertion (MCKI) as a baseline method, which retrieves relevant cultural knowledge from external memory using frozen MLLM representations. Experimental results indicate that current methods struggle to balance effective cultural adaptation with behavioral preservation, highlighting significant challenges in developing culturally aware AI. This work underscores the need for more responsible and adaptive MLLMs, providing a crucial resource for future research in mitigating cultural misalignment in artificial intelligence systems.
cs.AI updates on arXiv.org