FIRM: Fine-Grained Intra-Token Representation of Masks for Remote Sensing Reasoning Segmentation

arXiv:2608.13980v1 Announce Type: new Abstract: Reasoning segmentation requires multimodal large language models (MLLMs) to translate implicit instructions into precise pixel-level masks. MLLMs encode an image as visual tokens, each of which merges a group of image patches. In remote sensing images, small targets, thin structures, and adjacent instances can occupy different parts of the same visual token. Assigning a single binary mask label to such a token loses its internal spatial structure, causing nearby targets to merge and object boundaries to become coarse. To bridge this representational gap, we introduce FIRM, a Fine-grained Intra-token Representation of Masks. For each visual token, FIRM predicts a mask code that specifies an rimes r binary sub-cell pattern rather than a single foreground/background label. Given a target identified by the MLLM, the complete grid of mask codes is predicted in one mask pass. Fixed lookup converts the predicted codes into a discrete sub-cell mask, while marginalizing the code distribution yields a soft structural field. To further recover fine-grained boundaries within each sub-cell, we introduce a lightweight continuous renderer that refines this field using pre-merge visual features and image details. Across five reasoning and referring segmentation benchmarks on satellite and UAV images, FIRM achieves leading results, including 70.5/80.5 gIoU/cIoU on LaSeRS and a 3.0-point average gain on EarthReason. These results demonstrate the value of explicitly representing intra-token mask patterns for fine-grained MLLM segmentation.

Related

• Decomposed Attention Fusion in MLLMs for Training-Free Video Reasoning Segmentation
• Multimodal Large Language Models for Remote Sensing Image Understanding: Domain-Specific or General-Purpose?
• AnchorSeg: Language Grounded Query Banks for Reasoning Segmentation

Source: arXiv cs.CV | 2026-08-17