Safety
Evaluating LLM-Driven Summarisation of Parliamentary Debates with Computational Argumentation
arXiv:2604.19331v1 Announce Type: new Abstract: Understanding how policy is debated and justified in parliament is a fundamental aspect of the democratic process. However, the volume and complexity of
arXiv:2604.19331v1 Announce Type: new Abstract: Understanding how policy is debated and justified in parliament is a fundamental aspect of the democratic process. However, the volume and complexity of such debates mean that outside audiences struggle to engage. Meanwhile, Large Language Models (LLMs) have been shown to enable automated summarisation at scale. While summaries of debates can make parliamentary procedures more accessible, evaluating whether these summaries faithfully communicate argumentative content remains challenging. Existing automated summarisation metrics have been shown to correlate poorly with human judgements of consistency (i.e., faithfulness or alignment between summary and source). In this work, we propose a formal framework for evaluating parliamentary debate summaries that grounds argument structures in the contested proposals up for debate. Our novel approach, driven by computational argumentation, focuses the evaluation on formal properties concerning the faithful preservation of the reasoning presented to justify or oppose policy outcomes. We demonstrate our methods using a case-study of debates from the European Parliament and associated LLM-driven summaries.
Related
- Meet Dynamic Individual Preferences: Resolving Conflicting Human Value with Paired Fine-Tuning
- Does Welsh media need a review? Detecting bias in Nation.Cymru's political reporting
- ClimateCause: Complex and Implicit Causal Structures in Climate Reports
- Hierarchical Retrieval Augmented Generation for Adversarial Technique Annotation in Cyber Threat Intelligence Text
Source: arXiv cs.CL | 2026-04-22