On the Robustness of Vision-Language Models in Zero-shot Privacy Classification

arXiv:2510.09253v2 Announce Type: replace Abstract: Automatic systems for document understanding require multimodal models that accurately identify sensitive visual content, even in the presence of image degradations. Instruction-following large Vision-Language Models (VLMs) are expected to generalise across domains and tasks without requiring any specific adaptation. In this work, we systematically analyse whether VLMs can be used reliably for image privacy classification in a zero-shot setup. We evaluate and compare the classification performance of three open-source VLMs against purposely built models on two public, standard benchmarks. We assess robustness to image degradations caused by perturbations such as compression, light variations, and random noise, and analyse inference speed and parameter count to deploy VLMs in privacy-aware document processing pipelines. Our results show that large VLMs are robust to input perturbations but are less accurate (and much slower) than smaller privacy models. Scaling alone is not sufficient, highlighting the advantages of models specifically designed for privacy classification.

Related

• Auditing Data Leakage in Whole-Slide Image Multimodal Benchmarks
• Beyond Visual Evidence: Revealing and Mitigating Relational Privacy Leakage in Document MLLMs
• MultiPriv: Benchmarking Individual-Level Privacy Reasoning in Vision-Language Models

Source: arXiv cs.CV | 2026-08-20