Model Releases
BeyondMasks: Evaluating Causal and Physical Consistency in Video Object Removal
arXiv:2608.20107v1 Announce Type: new Abstract: Recent advances in generative video models have significantly improved visual realism in video object removal, yet evaluation protocols still focus on m
arXiv:2608.20107v1 Announce Type: new Abstract: Recent advances in generative video models have significantly improved visual realism in video object removal, yet evaluation protocols still focus on masked region fidelity, treating removal as local inpainting. In real scenes, object removal is a causal intervention: eliminating an object also requires removing its induced physical effects, such as shadows, reflections, illumination changes, translucency, and dynamic traces. Existing benchmarks lack aligned clean references or remain limited to simplified synthetic settings, preventing systematic evaluation of causal consistency. We introduce BeyondMasks, a paired benchmark for causally consistent video object removal, consisting of temporally aligned synthetic and real world video pairs with clean background references. The dataset spans diverse photometric, geometric, volumetric, and dynamic interactions, and supports both mask based and instruction driven editing. We further propose CORE, a structured vision language model based evaluation protocol that jointly measures object disappearance and after effect consistency, aligning more closely with human judgments than existing metrics. Benchmarking state of the art methods reveals systematic failures in removing secondary physical effects despite high masked region fidelity, exposing a gap between visual plausibility and causal correctness. BeyondMasks reframes video object removal as causal scene consistency rather than local reconstruction and provides a unified framework for its evaluation.
Related
- Learning Explicit Physical Parameter Control and Benchmarking for Video Generation
- Generative Action Tell-Tales: Assessing Human Motion in Synthesized Videos
- MechVerse: Evaluating Physical Motion Consistency in Video Generation Models
- Physics Question Scene Graph: Fine-grained Evaluation of Physical Plausibility in Text-to-Video Generation
- Each Judge Its Own Yardstick: Discovering Per-VLM Taxonomies for Physical Video Evaluation
Source: arXiv cs.CV | 2026-08-21