Labels have Human Values: Value Calibration of Subjective Tasks

arXiv:2601.06631v2 Announce Type: replace Abstract: Building NLP systems for subjective tasks requires one to ensure their alignment to contrasting human values. We propose the MultiCalibrated Subjective Task Learner framework (MC-STL), which clusters annotations into identifiable human value clusters by three approaches (similarity of annotator rationales, expert-value taxonomies or rater's sociocultural descriptors) and calibrates predictions for each value cluster by learning cluster-specific embeddings. We demonstrate MC-STL on several subjective learning settings, including ordinal, binary, and preference learning predictions, and evaluate it on multiple datasets covering toxic chatbot conversations, offensive social media posts, and human preference alignment. The results show that MC-STL consistently outperforms the baselines that ignore the latent value structure of the annotations, delivering gains in discrimination, value-specific calibration, and disagreement-aware metrics.

Related

• Coherence Maximization Improves Pluralistic Alignment
• Multi-Objective Exploration and Preference Optimization via Mutual Information
• Measuring Human Value Expression in Social Media Texts: Calibrated LLM Annotation and Encoder Transfer
• Meet Dynamic Individual Preferences: Resolving Conflicting Human Value with Paired Fine-Tuning

Source: arXiv cs.CL | 2026-09-01