Language Models Have Interpretable Narrative Mechanisms

arXiv:2610.12989v1 Announce Type: new Abstract: We investigate whether language models contain internal components that contribute to representing narrativity, the degree to which a text exhibits narrative properties. Across GPT-2-Small, Gemma-2-2B, and Llama-3.1-8B, we identify attention heads, MLP neurons, and sparse autoencoder features whose activations correlate with narrativity. Ablating selected neuron and feature sets increases next-token prediction loss more for high-narrativity than for low-narrativity texts, and reduces the narrativity of open-ended generations. On downstream benchmarks, the largest effects occur in story completion and narrative question answering, although they vary across models and languages; results on confabulation and hallucination benchmarks are mixed. These findings provide causal evidence that selected sets of narrative-associated components contribute to model behaviour, while their internal organisation, relation to narrative theory, and broader functions remain open questions.

Related

• On the Importance and Evaluation of Narrativity in Natural Language AI Explanations
• LitVISTA: A Benchmark for Narrative Orchestration in Literary Text
• Which India Survives Translation? Narrative Homogenisation Across Indian Oral Traditions in LLMs

Source: arXiv cs.CL | 2026-10-12