A Circuit for Plural Reference: How LLMs Represent and Retrieve Singular and Plural Entities

arXiv:2609.03687v1 Announce Type: new Abstract: Coreference resolution is an important task in contextual reasoning. In this paper, we investigate the mechanism for representing and retrieving singular and plural entities for plural reference. We use a combination of mechanistic interpretability and attention pattern analysis to study the process in which LLMs predict a pronoun to refer back to previously mentioned entities. Using a range of causal intervention techniques, we find a set of attention heads that are responsible for (1) representing coreference information in the input, (2) identifying entities that form a plural reference, (3) transferring the information to the component that is responsible for selecting the antecedents and predicting the pronoun. We also find that LLMs align with humans in preference for plural pronoun. Specifically, entities in a plural construction are more likely to be referred to as a plural entity if they are ontologically similar and are linked by the conjunction "and".

Related

• Findings of the Fifth Shared Task on Multilingual Coreference Resolution: Expanding Datasets for Long-Range Entities
• Mixing Mechanisms: How Language Models Retrieve Bound Entities In-Context
• CorPipe at CRAC 2026: Empty Nodes and Cross-Lingual Transfer in Multilingual Coreference Resolution
• Multilingual Coreference Resolution via Cycle-Consistent Machine Translation
• LLMBridge: An LLM Pipeline for End-to-end Referential Bridging Resolution in English

Source: arXiv cs.CL | 2026-09-04