Retell, Reward, Repeat: Reinforcement Learning for Narrative Theory-Informed Story Retelling

arXiv:2601.17226v3 Announce Type: replace-cross Abstract: Counterfactual storytelling exposes LLM shortcomings in causal reasoning, where few-shot prompting and SFT fail to produce logical and rational narratives. This case study introduces Retell, Reward, Repeat (RRR), a narrative theory-informed RL pipeline that teaches causal story structure. We extend an existing counterfactual stories dataset with human-annotated stages of narrative equilibrium, synthesising Structuralist Narratology with scalar narrativity. An aligned LLM-as-judge automates this framework to output reward signals, teaching the policy LLM to generate texts with causal narrative structure. Evaluations show that RRR-trained 7/8B LLMs outperform few-shot and SFT baselines in logic, rationality, and completeness, also aligning with a blind human preference subset. RRR offers a linguistically grounded, cost-effective post-training mechanism for LLM storytelling using a small, query-only dataset, showing the value of integrating linguistic theories into contemporary NLP.

Related

• Post-Training Large Language Models via Reinforcement Learning from Self-Feedback
• Reinforcement Learning with Temporal-Logic-Based Causal Diagrams
• Distilled Reinforcement Learning for LLM Post-training
• The Past Is Not Past: Memory-Enhanced Dynamic Reward Shaping

Source: arXiv cs.AI | 2026-10-15