After a Decade: Bringing Shadow Removal into the Real World with Agentic Training Data

arXiv:2609.38607v1 Announce Type: cross Abstract: Shadow removal looks nearly solved on established benchmarks, yet remains brittle in the real world. Models have advanced; the paired training data they rely on have barely changed in nearly a decade. The reason is simple: obtaining a shadow-free target requires removing the occluder while keeping the scene, camera, and illumination otherwise unchanged, making diverse paired data difficult to capture. Meanwhile, large shadow detection datasets already contain diverse real-world images and masks, but no shadow-free targets. To turn this abundant but incomplete data into paired supervision, we propose an offline agentic workflow combining physics-motivated generation, failure detection, feedback-driven retry, candidate selection, and deterministic correction. Using this workflow, we construct AgenticShadow, a dataset of 17,138 image-mask-target triplets spanning general scenes, faces, and remote sensing. Our construction workflow reduces Color Distribution Difference by 50.5% over previous shadow removal work, while training existing shadow removal models on AgenticShadow reduces cross-domain LAB RMSE by 19.7-37.5%.

Related

• MidTool: Mid-training Data Synthesis for Agentic Tool Use
• AutoVDC: Automated Vision Data Cleaning Using Vision-Language Models
• DR-Venus: Towards Frontier Edge-Scale Deep Research Agents with Only 10K Open Data

Source: arXiv cs.AI | 2026-10-01