Research
Zero-shot video highlight detection based on text descriptions and synthetic images
arXiv:2609.14790v1 Announce Type: new Abstract: Detecting video highlights, the most informative or engaging moments in a video, is important for applications such as video summarization and content r
arXiv:2609.14790v1 Announce Type: new Abstract: Detecting video highlights, the most informative or engaging moments in a video, is important for applications such as video summarization and content recommendation. We propose a zero-shot framework that combines CLIP, large language models (LLMs), and diffusion models. Given lightweight video metadata, such as a title or category, an LLM generates textual descriptions of likely highlight events. These descriptions are further converted into synthetic visual prototypes using a diffusion model. Textual and visual representations are matched to video frames using CLIP, enabling frame-level highlight detection without highlight annotations or dataset-specific training. Experiments on TVSum and SumMe demonstrate strong zero-shot performance, with particularly favorable results on TVSum. The proposed approach provides an effective framework for metadata-conditioned zero-shot video highlight detection.
Related
- Rethinking Auxiliary Modalities in Multimodal Zero-shot Anomaly Detection: From Semantic Fusion to Conditional Modulation
- Detecting Precise Hand Touch Moments in Egocentric Video
- FineHOI: Part-Aware Dense Representations for Zero-Shot Human-Object Interaction Detection
Source: arXiv cs.CV | 2026-09-15