Model Releases
StreamTTT: Reconciling Real-Time Perception and Long-Term Memory in Streaming VLMs
arXiv:2608.13416v1 Announce Type: new Abstract: Humans effortlessly perceive the present while remembering the past, yet streaming VLMs often trade off real-time perception against long-term memory. P
arXiv:2608.13416v1 Announce Type: new Abstract: Humans effortlessly perceive the present while remembering the past, yet streaming VLMs often trade off real-time perception against long-term memory. Prior work shows that shortening the context can sharpen current-scene perception at the expense of long-range recall. To reconcile these abilities, we introduce StreamTTT, which writes long-range history into online-updated fast weights outside the attention context. This leaves a short sliding key-value cache dedicated to recent evidence, mitigating attention dilution. We train StreamTTT jointly on offline long-video QA and a newly constructed real-time QA corpus. On OVO-Bench, StreamTTT-4B outperforms SimpleStream-4B by 1.4 points in real-time perception and 3.7 points in backward tracing. It also remains competitive with the larger SimpleStream-8B on the Real-Time Visual Understanding (RTVU) subset of StreamingBench. Our code will be released.
Related
- Video Active Perception: Effective Inference-Time Long-Form Video Understanding with Vision-Language Models
- VSAS-BENCH: Real-Time Evaluation of Visual Streaming Assistant Models
- StreamMeCo: Long-Term Agent Memory Compression for Efficient Streaming Video Understanding
- EPIC-Bench: A Perception-Centric Benchmark for Fine-Grained Embodied Visual Grounding in Vision-Language Models
- SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environments
Source: arXiv cs.CV | 2026-08-14