Hindsight Preference Optimization for Financial Time Series Advisory

arXiv:2604.23988v1 Announce Type: cross Abstract: Time series models predict numbers; decision-makers need advisory -- directional signals with reasoning, actionable suggestions, and risk management. Training language models for such predictive advisory faces a fundamental challenge: quality depends on outcomes unknown at prediction time. We bridge two ideas from reinforcement learning -- using information unavailable during execution to retrospectively generate training signal, and preference alignment -- and propose Hindsight Preference Optimization: observed outcomes let an LLM judge rank candidate advisories on dimensions that scalar metrics cannot capture, producing preference pairs for DPO without human annotation. We apply this to Vision-Language-Model-based predictive advisories on S&P 500 equity time series, demonstrated by a 4B model outperforming its 235B teacher on both accuracy and advisory quality.

Related

• Continuous-Utility Direct Preference Optimization
• Meta-Aligner: Bidirectional Preference-Policy Optimization for Multi-Objective LLMs Alignment
• Preference-Agile Multi-Objective Optimization for Real-time Vehicle Dispatching
• SPPO: Sequence-Level PPO for Long-Horizon Reasoning Tasks
• Data-efficient Targeted Token-level Preference Optimization for LLM-based Text-to-Speech

Source: arXiv cs.AI | 2026-04-28