Unmasking Propaganda: A Comparative Analysis of Masked and Causal Language Models

arXiv:2610.03077v1 Announce Type: new Abstract: Propaganda detection is an essential task in natural language processing (NLP), particularly in the context of manipulative political communications. However, identifying specific propaganda techniques presents a significant challenge due to their often subtle nature and reliance on context, making them difficult to distinguish from legitimate persuasive language. Propaganda often involves highlighting certain facts while downplaying or ignoring others to create a desired perception. This biased communication aims to influence attitudes, beliefs, or behaviors towards a particular cause or position. This paper explores advances in detecting propaganda techniques through a comparative analysis of modern language models, using the SemEval-2020 Task 11 dataset. We evaluated both masked language models (based on XLM-RoBERTa or DeBERTa V3) and causal models (from OpenAI, Google, Mistral, Anthropic and Meta), employing two prompting strategies: base and chain-of-thought prompting. Our results demonstrate improvements over state-of-the-art models, with the best-performing MLM achieving an F1 score of 63.18 in technique classification and the best causal model achieving 63.62. We also observed that certain models excel in specific techniques, such as loaded language and name-calling, while struggling with others like bandwagon and black-and-white fallacy. These findings suggest that fine-tuning, ensemble modeling, and the use of larger datasets can further enhance propaganda detection capabilities.

Related

• Cultural Adaptation in Large Language Models for Political Discourse
• AURORA: Asymmetry and Update-Induced Rotation for Robust Hallucination Detection in Large Language Models
• A Theoretical Analysis of Why Masked Diffusion Models Mitigate the Reversal Curse
• From Heads to Neurons: Causal Attribution and Steering in Multi-Task Vision-Language Models

Source: arXiv cs.CL | 2026-10-05