PPU-Bench: A Real-World Benchmark for Personalized Partial Unlearning in Vision Language Models
Researchers have introduced PPU-Bench, a novel benchmark designed to evaluate personalized partial unlearning in Multimodal Large Language Models (MLLMs). Existing benchmarks often rely on synthetic data or complete subject deletion, failing to address realistic, fine-grained requests for removing specific sensitive information. PPU-Bench utilizes 24,000 samples derived from the pre-existing knowledge of 500 public figures, testing models across Complete, Selective, and Personalized unlearning settings. The study highlights that current methods often suppress visual identity rather than factual knowledge and struggle with intra-subject factual boundaries. To address these challenges, the authors propose Boundary-Aware Optimization (BAO), a method that explicitly models forget-retain boundaries. Experimental results demonstrate that BAO effectively enforces these boundaries, improving the model's ability to remove target knowledge while preserving non-target facts and overall utility. This research contributes significantly to AI safety and privacy by providing robust tools for managing sensitive cross-modal information in vision-language systems.
Wire timeline
PPU-Bench: A Real-World Benchmark for Personalized Partial Unlearning in Vision Language Models
Researchers have introduced PPU-Bench, a novel benchmark designed to evaluate personalized partial unlearning in Multimodal Large Language Models (MLLMs). Existing benchmarks often rely on synthetic data or complete subject deletion, failing to address realistic, fine-grained requests for removing specific sensitive information. PPU-Bench utilizes 24,000 samples derived from the pre-existing knowledge of 500 public figures, testing models across Complete, Selective, and Personalized unlearning settings. The study highlights that current methods often suppress visual identity rather than factual knowledge and struggle with intra-subject factual boundaries. To address these challenges, the authors propose Boundary-Aware Optimization (BAO), a method that explicitly models forget-retain boundaries. Experimental results demonstrate that BAO effectively enforces these boundaries, improving the model's ability to remove target knowledge while preserving non-target facts and overall utility. This research contributes significantly to AI safety and privacy by providing robust tools for managing sensitive cross-modal information in vision-language systems.
cs.AI updates on arXiv.org