Boundary Voting Network for Ambiguity-Aware Timestamp-Supervised Action Segmentation

arXiv:2609.08167v1 Announce Type: new Abstract: Timestamp-supervised action segmentation aims to segment and classify actions in untrimmed videos with a random frame annotated per action. Precisely localizing action boundaries from timestamp annotations is crucial for this setting, as it enables generating framewise pseudo-labels and applying the well-explored fully-supervised training. However, prevailing methods struggle with intrinsic uncertainty in boundary localization due to less discriminative features in action-transiting regions. This imprecise boundary estimation significantly reduces the stability and reliability of the generated pseudo-labels in ambiguous action-transiting regions, consequently resulting in performance deterioration of the trained segmentation models. In our paper, we introduce the boundary voting network that mitigates feature ambiguity by hierarchically propagating video-level global prior knowledge into local action-transiting regions. By generating key action representations as votes throughout the video and targeting action-transiting regions, all votes collaboratively contribute to action-transiting feature enhancement and boundary localization refinement. Extensive experiments demonstrate the effectiveness of our method on GTEA, 50Salads, and Breakfast datasets.

Related

• Improving Temporal Action Segmentation via Constraint-Aware Decoding
• Boundary-Centric Active Learning for Temporal Action Segmentation
• IMPACT-Scribe: Interactive Temporal Action Segmentation with Boundary Scribbles and Query Planning
• OnPoint: Offline-to-Online Multi-Level Distillation for Point-Supervised Online Temporal Action Localization

Source: arXiv cs.CV | 2026-09-09