BenchPreS: A Benchmark for Context-Aware Personalized Preference Selectivity of Persistent-Memory LLMs

arXiv:2603.16557v2 Announce Type: replace-cross Abstract: Large language models (LLMs) increasingly store user preferences in persistent memory to support personalization across interactions. However, in third-party communication settings governed by social and institutional norms, some user preferences may be inappropriate to apply. We introduce BenchPreS, which evaluates whether memory-based user preferences are appropriately applied or suppressed across communication contexts. Using two complementary metrics, Misapplication Rate (MR) and Appropriate Application Rate (AAR), we find even frontier LLMs struggle to apply preferences in a context-sensitive manner. Models with stronger preference adherence exhibit higher rates of over-application, and neither reasoning capability nor prompt-based defenses fully resolve this issue. These results suggest current LLMs treat personalized preferences as globally enforceable rules rather than as context-dependent normative signals.

Related

• Towards Natural Personalization: Evaluating Long-Horizon Preference Following in Personalized User-LLM Interactions
• TSUBASA: Improving Long-Horizon Personalization via Evolving Memory and Self-Learning with Context Distillation
• AdaMem: Adaptive User-Centric Memory for Long-Horizon Dialogue Agents

Source: arXiv cs.CL | 2026-10-15