GenState-AI: State-Aware Dataset for Text-to-Video Retrieval on AI-Generated Videos

arXiv:2603.14426v2 Announce Type: replace Abstract: Existing text-to-video retrieval benchmarks often permit matching from static appearance, making it difficult to determine whether models capture state changes and final outcomes. We introduce GenState-AI, a benchmark comprising 956 AI-generated videos and 195 verified state-change queries across Toy, Cartoon, Home, and Manipulation domains. Its construction uses prompt-controlled variants that target either conflicting temporal/state outcomes or semantic substitutions. Following human verification, all valid videos contribute to subset-level retrieval pools, while complete query-specific triplets support diagnostic hard-negative ordering. We evaluate VCF-Lik, Qwen3-VL, and LanguageBind using both standard Recall@K and triplet-based consistency metrics. Qwen3-VL (8B) achieves the best macro-average R@1 and triplet consistency, while LanguageBind obtains the highest semantic discrimination accuracy. However, stronger retrieval does not consistently yield correct hard-negative ordering, and Qwen3-VL and LanguageBind remain weak on Manipulation triplets, where VCF-Lik performs substantially better. GenState-AI therefore provides a controlled diagnostic testbed for distinguishing global retrieval quality from fine-grained temporal and semantic grounding in AI-generated videos.

Related

• VideoVerse: Does Your T2V Generator Have World Model Capability to Synthesize Videos?
• ARK: A Dual-Axis Multimodal Retrieval Benchmark along Reasoning and Knowledge
• EgoExo-Next:Benchmarking Vision-Language Models on Visual-Option Next-State and Cross-View Reasoning
• VideoZeroBench: Probing the Limits of Video MLLMs with Spatio-Temporal Evidence Verification
• SVCBench: A Streaming Video Counting Benchmark for Spatial-Temporal State Maintenance

Source: arXiv cs.CV | 2026-10-15