Preference Is Not Intervention: The Structure and Stability Boundaries of Reader-Specific Evidence Utility

arXiv:2608.17781v1 Announce Type: new Abstract: ML systems increasingly condition decisions on downstream model identity, but this is useful only if model-specific differences form reusable structure rather than input-local interactions. We test this in retrieval-augmented generation (RAG), where evidence utility can be measured under controlled interventions. Holding query, evidence, task, scoring, and intervention fixed, nine readers disagree on effect sign in 33% of jointly affected cells; readerimesquery interaction explains 29.8% of utility variance versus an 8.4% permutation null; and self-selected evidence improves F1 by +0.031 (t=3.39). We then ask the sharper question: which components of this heterogeneity are stable reader properties across queries? Separating three measurable objects---evidence activity, ordinal preference, and conditional signed direction---we find ordinal reader geometry stable across four independent settings (split-half rho=0.60--0.83): leave-one-out interventions, PRISM preferences, RAMDocs, and RAGuard. Signed geometry is task-bounded: weak in open-ended QA (0.14, 0.35), especially for misleading and irrelevant evidence, but strong in binary fact-checking (0.75) with no significant ordinal gap, though still below its sparsity-matched ceiling. Sparsity, decoding noise, and metric artifacts do not explain the main ordinal--signed gap. Finally, stable ordinal similarity fails to predict cross-reader intervention transfer (oracle-distance rho=-0.27; regret reliability -0.28). Reader-specific utility exists, but preference is not intervention: stable ranking similarity does not license transfer of help/harm decisions.

Source: arXiv cs.CL | 2026-08-19