Information-Guided Frontier Decoding: Contextual Utility-Driven Commitment in dMLLMs

arXiv:2608.26641v1 Announce Type: new Abstract: Decoding quality in diffusion multimodal language models (dMLLMs) depends heavily on the order in which masked tokens are committed. Existing confidence-based strategies prioritize locally easy tokens, but confidence does not necessarily reflect contextual usefulness. As a result, structurally easy tokens such as punctuation may be committed before informative semantic anchors, weakening context propagation and increasing error accumulation. We propose Information-Guided Frontier Decoding (IGFD), a training-free decoding strategy that ranks candidates using token confidence, neighborhood uncertainty, and structural commitment risk. IGFD encourages early commitment of reliable semantic anchors while delaying fragile structural tokens, improving contextual support during decoding. A dynamic candidate frontier further constrains token selection to locally expandable regions under the same decoding budget. The method requires no additional training, auxiliary models, or extra forward passes. Experiments across multimodal understanding, reasoning, grounding, and hallucination benchmarks show that IGFD consistently outperforms existing decoding strategies across the majority of benchmarks and diffusion MLLM backbones under identical decoding budgets.

Related

• Context-Aware Cluster Decoding: Semantic Anchor-Driven Coherence in dMLLMs
• NAVIRA: Decoupled Stochastic Remasking for Masked Diffusion Language Models
• Self-Pruned Key-Value Attention: Learning When to Write by Predicting Future Utility
• When to Plan, When to Polish: Noise Level as a Granularity Axis for Diffusion Language Models

Source: arXiv cs.CL | 2026-08-28