Knowledge Poisoning Attacks on Medical Multi-Modal Retrieval-Augmented Generation
Researchers have introduced M³Att, a novel knowledge-poisoning framework targeting medical multi-modal Retrieval-Augmented Generation (RAG) systems. While RAG enhances Large Language Models (LLMs) with expert knowledge, it remains vulnerable to adversarial data injection. Previous attack models unrealistically assumed adversaries knew user queries in advance. In contrast, M³Att operates with limited database knowledge, injecting covert misinformation into textual data and using paired visual data as query-agnostic triggers to manipulate retrieval probabilities. The framework exploits the inherent ambiguity in medical diagnoses to bypass LLM self-correction mechanisms, which typically fix explicit factual errors. By introducing imperceptible perturbations to visual inputs, the attack ensures poisoned content is retrieved and generated. Experiments across five LLMs and datasets demonstrate that M³Att consistently produces clinically plausible yet incorrect diagnostic outputs, significantly undermining system reliability. This study highlights critical security vulnerabilities in AI-driven medical applications and provides open-source code to facilitate further research into defensive strategies against such sophisticated poisoning attacks in multimodal environments.
Wire timeline
Knowledge Poisoning Attacks on Medical Multi-Modal Retrieval-Augmented Generation
Researchers have introduced M³Att, a novel knowledge-poisoning framework targeting medical multi-modal Retrieval-Augmented Generation (RAG) systems. While RAG enhances Large Language Models (LLMs) with expert knowledge, it remains vulnerable to adversarial data injection. Previous attack models unrealistically assumed adversaries knew user queries in advance. In contrast, M³Att operates with limited database knowledge, injecting covert misinformation into textual data and using paired visual data as query-agnostic triggers to manipulate retrieval probabilities. The framework exploits the inherent ambiguity in medical diagnoses to bypass LLM self-correction mechanisms, which typically fix explicit factual errors. By introducing imperceptible perturbations to visual inputs, the attack ensures poisoned content is retrieved and generated. Experiments across five LLMs and datasets demonstrate that M³Att consistently produces clinically plausible yet incorrect diagnostic outputs, significantly undermining system reliability. This study highlights critical security vulnerabilities in AI-driven medical applications and provides open-source code to facilitate further research into defensive strategies against such sophisticated poisoning attacks in multimodal environments.
cs.AI updates on arXiv.org