AudioTETRIS: Token Elimination via Transitional Importance Signals for Efficient Audio LLMs

arXiv:2610.13600v1 Announce Type: cross Abstract: Large Audio Language Models (LALMs) achieve strong performance on speech-text tasks, but standard audio encoders produce hundreds of tokens for even short audio clips. Because self-attention scales quadratically in sequence length, this inflates inference-time compute substantially. Current training-free token compression methods prune or merge tokens based solely on coarse attention or feature similarity. However, treating audio tokens as generic, isotropic features smooths over spectral transitions, destroying fine-grained phonetic transitions essential for decoding speech. In contrast, speech signals exhibit highly structured temporal patterns where information density peaks at acoustic transitions. We propose AudioTETRIS, a training-free, two-stage framework that uses the audio encoder's representation geometry as an acoustic prior for token importance before applying attention-based compression. Instead of discarding encoder structure after the encoding stage, AudioTETRIS propagates this prior directly into the LLM decoder to compute a per-token fused score. This score drives progressive top-k token selection across decoder layers without requiring additional training or auxiliary models. Evaluated on Qwen2-Audio-7B, Phi-4-Multimodal, and Audio Flamingo 3, AudioTETRIS retains near-uncompressed performance on several speech tasks using only 25% of audio tokens, matches or surpasses full-token accuracy at moderate compression rates, and consistently outperforms existing training-free baselines.

Related

• TokenButler: Token Importance is Predictable
• TriSP: Tri-Signal Structured Pruning for Large Language Models
• Hybrid JIT-CUDA Graph Optimization for Low-Latency Large Language Model Inference
• DistillCache: KL-Guided Adaptive KV-Cache Eviction for Memory-Efficient LLM Inference

Source: arXiv cs.AI | 2026-10-12